Out

mysqldump –databases database_name -u username -p > dumpfile.sql

In

mysql -u username -p database_name < dumpfile.sql

Find the reference here.

15/02/10: Here is another variation on getting stuff out:

mysqldump db_name table1 table2 table3 -u username -p > dumpfile.sql

For the zipfile manipulation thanks go to this post and the original rubyzip API.

So, first make the HTTP request:

http = Net::HTTP.new(”localhost”, 10000)
http.read_timeout=6000
response = http.get(’csvserver/download/+ filename)

#check that the request was OK
if response.response.class == Net::HTTPOK

#check what the response contains

if response.content_type == ‘application/zip’

#make a new file to hold the zipped data

uuid = UUIDTools::UUID.random_create.to_s

#you might need to install the uuidtools gem

File.makedirs(RAILS_ROOT + “/” + “filestore” + “/” +filename)
uf = File.open(RAILS_ROOT + “/” + “filestore” + “/” + filename  + “/” + uuid+ “.zip”,”w”)

#write the content in the response out to the zip file
uf.write(response.body)
uf.close

After this the metadata is created and written to a string called metadata which can then be added to the zip we just created from the http response.  This next line adds the metadata string to the zip inside a file called metadata.txt.

Zip::ZipFile.open(RAILS_ROOT + “/” + “filestore” + “/” + filename  + “/” + uuid+ “.zip”, Zip::ZipFile::CREATE) {|zip| zip.get_output_stream(”metadata.txt”) { |f| f.puts metadata}}

We can then send the completed file back to the requester

send_file RAILS_ROOT + “/” + “filestore” + “/” + filename  + “/” + uuid+ “.zip”, :filename => “something” + “.zip”, :content_type => @archive.content_type, :disposition => ‘attachment’

I have also been looking at using Ruby to do all the CSV file creation but so far speed is proving to be an issue.  However, I found the following link which mentions several libraries.  I am now looking at using a combination of FasterCSV and CSVScan (also available as a gem).  This enables me to write incredibly complex code like this (don’t you just love ruby’s blocks!):

CSVScan.scan(infile) { |row| line = FCSV.parse(row[0], :col_sep => “\t”); new_row = Array.new; if i==0: variable_hash[key].each {|var| pos.push(line[0].rindex(var))} end; i=i+1; pos.each {|col| val = line[0][col];new_row.push(val)}; csv_arr.push(new_row)}

Yeah, it scared me as well!!  It parses a CSV file for several headers and then pulls out the columns which correspond to these headers and adds them into an array.  In MethodBox these files can have 2000 header (ie columns) and 20000 rows so you can see why speed is important when only pulling out a subset of these.

We can then use FasterCSV to write out the new CSV file:

FasterCSV.open(outfile, “w”) do |csv_file| csv_arr.each {|csv| csv_file << csv} end

Easy!!  Only took 2 hours.  So that’s about 4 hours (and 2GB+ downloads) for the snow leopard upgrade so far!

Also found this script for installing ImageMagick.  I couldn’t get it to work but maybe you can.

Writing complex apps in html is certainly an interesting challenge and after having created the first version of what we now call methodbox I can see why frameworks like Google Web Toolkit and flex are so appealing.  Having to insert hidden objects into the html so the application can remember them between clicks gets a bit tedious after a while! However, we overcame a few hurdles along the way and here are some things I found useful:

Things I needed to know about Rails but was too much of a newby to find out from the books

1. routes.rb.  How to specify routes correctly.  Part of the functionality of methodbox is selecting multiple surveys and searching them.  I couldn’t get the app to return to the correct route after doing the search. Of course, I had not specified the route correctly.  So, if you are returning a collection you need something like this:

map.resources:surveys, :member => {:download => :get}, :collection => {:add_to_pseudo_cart => :get,  :data => :get, :datagrid => :get,:hide_info => :get, :more_info => :get, :search_variables => :get,:sort_variables => :get }  do |survey|

2. Check box tag.  Use check_box_tag to allow you to select multiple objects in the page and send them across inside a hash to the controller.

<%= check_box_tag “entry_ids[]”, item.id,false, :class=>’survey_checkbox’ %>

where item is the object which you want to select or not, false means unchecked when created

3. Selecting, unselecting checkboxes programatically.  Lets say you want a gmail style select all, none (and an ‘invert’ selection - ie. select those not selected and vice versa).  Use link to function and some javascript like this:

<%= link_to_function ‘All’, “$$(’input.survey_checkbox’).each(function(checkbox) { checkbox.checked = true; });” %>

<%= link_to_function ‘None’, “$$(’input.survey_checkbox’).each(function(checkbox) { checkbox.checked = false; });” %>

<%= link_to_function ‘Invert selection’, “$$(’input.survey_checkbox’).each(function(checkbox) { if (checkbox.checked == true) {checkbox.checked = false;} else {checkbox.checked = true; } });” %>

where ’survey_checkbox’ is the class name of the check_box_tag (see 2 above).

4. XML creation and parsing.  Tried the various libraries but found libxml the fastest.  I needed to parse some fairly big (1 meg+) xml files which contained the metadata for the surveys we are using.  These surveys take the form

<metadata year=“2000″>
<variable>
<name>difbpc1</name>
<description>BP Probs: No problems taking blood pressure</description>
<information>
<info>
<Value>-9</Value>
<Label>Refusal (8)</Label>
</info><info><Value>-8</Value>
<Label>Don’t know (9)</Label>
</info></information>
<MissingValues>-99  thru -1</MissingValues>
</variable>
….
….
</metadata>

Here is some sample code to parse (some of) this using libxml (don’t forget your require ‘xml’ bit):

parser = XML::Parser.file(’metadata.xml’)
doc = parser.parse

nodes = doc.find(’//metadata/variable’)

nodes.each do |node|
namenode = node.find(’child::name’)
namecontent = namenode.first.content
variable.name = namecontent

descnode = node.find(’child::description’)
desccontent = descnode.first.content

catnode = node.find(’child::category’)
catcontent = catnode.first.content

dernode = node.find(’child::derivation’)
dercontent = dernode.first

dertype = dercontent.find(’child::type’)
dertypecontent = dertype.first.content

dermethod = dercontent.find(’child::method’)
dermethodcontent = dermethod.first.content

infonode = node.find(’child::information’)
infocontent = infonode.first.content

end

5. gotAPI.  I used this online resource ‘a lot’.  A superb go to when the books are confusing or hard to navigate.

So, a few things which confused me and one major resource you really need.  See you on the Rails.

• Remember your Cross Domain policy or your client will not be able to talk to your server
• When adding custom MXML modules programatically there is no guarantee that any of the components inside will exist.  Can lead to lots of annoying null pointer exceptions.  You might have to check that they are not null and create them in actionscript.  I have found that doing something with the parent module eg. adding to a tab navigator forced the child components into life.  Setting the creationPolicy to ALL seems to have no affect.  I look forward to finding out the correct way to do this because I have found little to help on the web.
• When downloading files using FileReference you have to add an event listener to handle the IOErrorEvent.IO_ERROR or else you get a #2038 IO Error
• To add an icon to a button that has been created in ActioScript you have to declare the icon you want to use like this:

1. [Embed(source=’/assets/analysis_small.png’)]
   [Bindable]
   private var analysisImage:Class;
2. Assign using code: analysisButton.setStyle(”icon”, analysisImage);

• Events seem to pass from child to parent but not the other way.  When adding event listeners you need to add them to the level above the originator of the event.  Has meant all sorts of weird things like systemManager.addEventListener and Application.application.addEventListener.  Now, I might be getting this wrong but no amount of googling has led to a sensible answer.

Overall, FLEX seems great for UIs which are not reacting to data changes but needs a bit of tweeking by Adobe to make it a real power users language of choice.  Doing things like grids which react to check box clicks needs quite a bit of code.  There also seems to be some fundamental UI components missing eg. dockable drag and drop panels.  Sure, there is usually some open source solution available or you can write your own but it would be nice if the core API had these things.

read.dta(”filename”, convert.dates = TRUE, tz = NULL, convert.factors = TRUE, missing.type = FALSE, convert.underscore=TRUE, warn.missing.labels=TRUE)

You can then connect to them remotely using the Remote Java Application debug configuration in eclipse.  You can then use the eclipse debugger to step through any tricky code.

The application will start with “Listening for transport dt_socket at address: 8993″ and you connect to it through this port. Inside eclipse you can add any jars, local projects etc. to the debugger as it steps into new classes and you can then view the code (and the application) as it runs.

response.getEntity().getStream().close();

or

response.getEntity().write(System.out);

Otherwise the client keeps trying to send to the server but can’t get an open socket to get there.  Another problem solved!

inputs.put(”fileUrl”, LocalworkerTranslator.class.getResource( “/AAC4_HUMAN.sp”).getFile());

you have to ensure that the absolute path is correct by doing

URI uri = LocalworkerTranslator.class.getResource(”/AAC4_HUMAN.sp”) .toURI();

File newFile = new File(uri);

inputs.put(”fileUrl”, newFile.getAbsolutePath());

Also, you can’t expect checks for “\n” style line breaks to work in Windows because they insert things like “\r\n” instead.  So , you should use System.getProperty(”line.separator”) instead.