Friday, April 23, 2010

Using annotate_models plugin

download or install annotate_models plugin (http://github.com/ganeshprasad/annotate_models) into vendor/plugins folder.
Now whenever you run the command rake annotate_models it will update the models with the schema.
Schema is also defined in ActiveRecord models whenever rake db:migrate is done.


Add a comment summarizing the current schema to the top or bottom of each of your…
* ActiveRecord models
  * Fixture files
  * Tests and Specs
  * Object Daddy exemplars
  * Machinist blueprints
The schema comment looks like this:
# == Schema Info
   #
   # Table name: line_items
   #
   #  id                  :integer(11)    not null, primary key
   #  quantity            :integer(11)    not null
   #  product_id          :integer(11)    not null
   #  unit_price          :float
   #  order_id            :integer(11)
   #

    class LineItem < ActiveRecord::Base
      belongs_to :product
     . . .

Thursday, April 22, 2010

Reading huge csv or xls file and importing the data into database tables.

To implement this we need to gems fastercsv (gem install fastercsv) and ar-extensions ( gem install ar-extensions)

FasterCSV is intended as a replacement to Ruby‘s standard CSV library. It was designed to address concerns users of that library had and it has three primary goals:
  1. Be significantly faster than CSV while remaining a pure Ruby library.
  2. Use a smaller and easier to maintain code base. (FasterCSV is larger now, but considerably richer in features. The parsing core remains quite small.)
  3. Improve on the CSV interface. 
Advantages:

  • FasterCSV has a stricter parser and will throw MalformedCSVErrors on problematic data.
  • FasterCSV has a less liberal idea of a line ending than CSV. What you set as the :row_sep is law.
  • CSV returns empty lines as [nil]. FasterCSV calls them [].
  • FasterCSV has a much faster parser. 

ar-extensions

ActiveRecord::Extensions provides extensions to:
  • mass import data
  • convert ActiveRecord models and arrays to CSV format
  • work with temporary tables
  • control foreign keys
  • to include extensible features for better and custom finder support
Sample code to implement :
require 'rubygems'
require 'fastercsv'
require 'ar-extensions'
table_columns = [:attribute1,:attribute2, :attribute3, :attribute4, :attribute5 ]
table_data = Array.new


FasterCSV.foreach("#{RAILS_ROOT}/public/data/yourcsvfile.csv") do |row|
  row.delete_at(14)   #this can be used incase you want to delete any unnecessary column from any row of csv data

  table_data << row
end
options = { :validate => false }
CareHomeHq.import table_columns,table_data

Here insert of data will also be faster because we are not creating a new instance of CareHomeHq everytime a new row is fetched. Instead all the rows are saved in an array first and then we use ar-extensions to import the data.


PARSING FROM TEMP FILE
In case you want to parse from temp file then all you need to do is
  @parsed_file = FasterCSV.parse(params[:data_import][:file].read).each do |row|   
       puts row.inspect
      #your code here     
   end

Monday, April 5, 2010

Conditional validations in rails

By default, validations will take place every time the model is saved. Sometimes you only want a validation to happen when certain conditions are met. See how to do that in this episode.

# models/user.rb
validates_presence_of :password, :if => :should_validate_password?
validates_presence_of :country
validates_presence_of :state, :if => :in_us?
attr_accessor :updating_password

def in_us?
country == 'US'
end

def should_validate_password?
updating_password || new_record?
end

# in controller
@user.updating_password = true
@user.save

# or...
@user.save(false)

Monday, March 1, 2010

Ruby: Large xml files Parsing With SAX

SAX is an event-driven parser for XML.

It sequentially reads the xml and generates special events. So, if you want to use SAX, you should implement the code to handle them. It's quite different from the DOM model, where the whole xml is parsed and loaded in an tree.

The Ruby XML Library

The Ruby core library has a built-in XML parser (both DOM and SAX) called REXML, but it's terribly slow, it's highly advisable to use libxml. It's a binding to the popular library from Gnome and it was released as gem.

Installing libxml is simple by running the following command.

gem install libxml-ruby

Refer the following example which reads a large xml file and inserts the data into the corresponding database table.

require 'xml/libxml'
CombinationPackInd.benchmark("Truncating look up tables and inserting new records") do
#truncate combination_pack_ind table
ActiveRecord::Base.connection.execute("TRUNCATE combination_pack_ind")
#truncate combination_prod_ind table
ActiveRecord::Base.connection.execute("TRUNCATE combination_prod_ind")
############################# ---TRUNCATING ENDS--- ########################

class Lookuphandler
include XML::SaxParser::Callbacks
def on_start_element_ns (name, attributes, prefix, uri, namespaces)
@tag_name = name
@main_tag_name = "" if @main_tag_name.nil?
@main_tag_name = case name
when "COMBINATION_PACK_IND"
then "COMBINATION_PACK_IND"
when "COMBINATION_PROD_IND"
then "COMBINATION_PROD_IND"
else
@main_tag_name
end
end

def on_end_element_ns (name, prefix, uri)
@end_element = name


#saving combination pack ind
if @main_tag_name == "COMBINATION_PACK_IND"
if @tag_name == "CD"
@cd = @value
elsif @tag_name == "DESC"
@desc = @value
end

if @end_element == 'INFO'
@comb_pack_ind = CombinationPackInd.new(:DESC => @desc)
@comb_pack_ind.CD = @cd
@comb_pack_ind.save
# clear the variables so that it wil not carried to the next instance.
@cd, @desc = ""
end
end
#saving combination prod ind
if @main_tag_name == "COMBINATION_PROD_IND"
if @tag_name == "CD"
@cd = @value
elsif @tag_name == "DESC"
@desc = @value
end

if @end_element == 'INFO'
@comb_prod_ind = CombinationProdInd.new(:DESC => @desc)
@comb_prod_ind.CD = @cd
@comb_prod_ind.save
# clear the variables so that it wil not carried to the next instance.
@cd, @desc = ""
end
end

##################################### CLEARING VARIABLE CONTAINING TAG INFO AFTER INSERTING RECORD ##################################
@tag_name = ""
@main_tag_name = "" if @end_element == ( "COMBINATION_PROD_IND" || "COMBINATION_PACK_IND")
end

def on_characters(s)
@value = s
end

end
################# ---LOOKUP PARSING BEGINS--- #########################################################
file_path = RAILS_ROOT + "/data/file.xml"
parser = XML::SaxParser.file(file_path)
parser.callbacks = Lookuphandler.new
parser.parse
end

Thursday, February 25, 2010

Using REXML to read small xml files and load into database table

Save the following code in lib folder of your application as read_xml_file.rb . using command prompt run " ruby script/runner lib/read_xml_file.rb". The following code is possible to read only small xml files ( usually in kbs).


require 'rexml/document'
require 'rubygems'
include REXML
#def import_xml(tag)
file= File.new(RAILS_ROOT + "/file.xml")
doc= Document.new(file.read)

Ingredient.benchmark("Truncating ingredient table and inserting new records") do
#truncate ingredient table
ActiveRecord::Base.connection.execute("TRUNCATE ingredient")
# read table contents
XPath.each( doc, "//INGREDIENT_SUBSTANCES//ING" ){|ingredient|
ingredient_params = {}
#if there are more than one attributes for table then create a hash.
ingredient.children.each do |child|
key = "#{child.name}".to_sym
if key.to_s == "ISID"
@isid_value = child.text
else
ingredient_params[key] = child.text
end
#@desc_value = child.text if key.to_s == "DESC"

end
#create new instance
ingredient = Ingredient.new(ingredient_params)
#set the primary key if mass assigning of primary key is not possible.
ingredient.ISID = @isid_value
#save
ingredient.save
}
end

TRUNCATE All Tables in a Ruby on Rails

There might be a case where you need to delete all the rows from all the tables in the database to end up with just the structure(a bare DB). The right thing to do is to run the TRUNCATE command on all the tables. It will delete all the data and also reset the auto increment value. But the only current way to get a bare DB in rails now is to run rake db:reset. This will run rake db:drop, rake db:create and rake db:migrate. The downside of this is that if you have a reasonable number of migrations this command can take quite some time to run which is really not efficient when you are iterating fast. Hence following rake task will
TRUNCATE all the tables.

namespace :db do
task :load_config => :rails_env do
require 'active_record'
ActiveRecord::Base.configurations = Rails::Configuration.new.database_configuration
end

desc "Create Sample Data for the application"
task(:truncate => :load_config) do
begin
config = ActiveRecord::Base.configurations[RAILS_ENV]
ActiveRecord::Base.establish_connection
case config["adapter"]
when "mysql"
ActiveRecord::Base.connection.tables.each do |table|
ActiveRecord::Base.connection.execute("TRUNCATE #{table}")
end
when "sqlite", "sqlite3"
ActiveRecord::Base.connection.tables.each do |table|
ActiveRecord::Base.connection.execute("DELETE FROM #{table}")
ActiveRecord::Base.connection.execute("DELETE FROM sqlite_sequence where name='#{table}'")
end
ActiveRecord::Base.connection.execute("VACUUM")
end
rescue
$stderr.puts "Error while truncating. Make sure you have a valid database.yml file and have created the database tables before running this command. You should be able to run rake db:migrate without an error"
end
end
end


Just create a file named db_truncate.rake in your lib/tasks directory
with this code in it. Save the file and then run rake db:truncate. Your
database now should have no data now. Before you run the task make sure
that you have a valid database.yml file and have created the database
tables before running this command. You should be able to run rake
db:migrate without an error

In case you want to truncate a single table from database all you have to do is
ActiveRecord::Base.connection.execute("TRUNCATE table_name")



Monday, February 15, 2010

Avoid validations on create or update in rails

There might be several situations where we might have to avoid the default validations defined in the model when we update or create a record.
a.)One solution to avoid validation is by doing something like this
@something = Something.find(params[:id])
@something.column_save = 'your data'

respond_to do |format|
if @something.save(false) #save without validation
b.)Another solution is to write a separate method to validate in the model
please see the example below
validate :validates_uniqueness_of_name

def validates_uniqueness_of_name
return if deleted == true
num_duplicates = self.class.count(:conditions => ["name = ? AND deleted = ?",self.name, false])
if num_duplicates > 0
errors.add(:name, :taken)
end
end

How to delete a many-to-many association with Rails

One solution is to create a new model for the association. It should be the case
if you add attributes to the association (because push_with_attributes is now deprecated).
You can then simply find the association given the ids of your linked object and call destroy.

However, when you don't have any attribute in your liaison, the has_and_belongs_to_many
is nicer to work with. (you don't need a rails model for the liaison.)
Here is a link to the methods has_and_belongs_to_many adds where we can read :


"collection.delete(object, …) - removes one or more objects from the
collection by removing their associations from the join table.
This does not destroy the objects."

Let's assume we dispose of 2 models 'Post' and 'Category' with a N-N association :

class Post < ActiveRecord::Base
has_and_belongs_to_many :categories
end

class Category < ActiveRecord::Base
has_and_belongs_to_many :posts
end

To delete an association (remove a post from a category) you can use this method :

  def remove_post_from_category
post = Post.find(params[:post][:id])
category = post.categories.find(params[:category][:id])

if category
post.categories.delete(category)
end

end

This function will destroy the association but won't destroy the category.

You can also removes all the categories from the posts by using :


collection.clear - removes every object from the collection.
This does not destroy the objects.
In our case its

post.categories.clear

Saturday, September 26, 2009

Reliance Netconnect Broadband connection in Ubuntu 8.10 desktop edition

Being a programmer in always wanted internet connection in my Ubuntu. I was wondering if it is possible to do that before i found this blog http://himanshuonweb.blogspot.com. Thanks to Himanshu.
I followed those instructions and i was successfully able to connect to internet.
Follow these steps below:
1- Start Ubuntu ensuring USB modem is not connected and configure the file /etc/wvdial.conf
sudo gedit /etc/wvdial.conf


paste the following lines:


[Dialer Defaults]

Init1 = ATZ

Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0

Stupid Mode = 1

Modem Type = USB Modem

ISDN = 0

Phone = #777

New PPPD = yes

Modem = /dev/ttyUSB0

Username =

Password =
Problem while saving?? I believe you do not have permission to save contents. You can manually set the permission to write contents by typing sudo chmod 777 /etc/youfilename (in this case your file name is wvdial.conf). You might also want to set "write" permissions where you are not able to modify the file in the steps below
2- Plug-in USB modem, your /var/log/messages will display something like:


Aug 16 19:35:57 abc-laptop kernel: [ 111.532151] usb 4-1: new full speed USB device using uhci_hcd and address 2
Aug 16 19:35:57 abc-laptop kernel: [ 111.744188] usb 4-1: configuration #1 chosen from 1 choice
Aug 16 19:35:57 abc-laptop kernel: [ 111.747614] usbserial_generic 4-1:1.0: generic converter detected
Aug 16 19:35:57 abc-laptop kernel: [ 111.748339] usb 4-1: generic converter now attached to ttyUSB0
Aug 16 19:35:57 abc-laptop kernel: [ 111.909454] usbcore: registered new interface driver libusual


3- Run
lsusb
whose output will be like:

us 007 Device 002: ID 04f2:b008 Chicony Electronics Co., Ltd
Bus 007 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 002: ID 12d1:142b Huawei Technologies Co., Ltd.
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 001 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 006 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 005 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub


4- Add a module for the above listed Huawei device, as:

sudo modprobe usbserial vendor=0x12d1 product=0x142b


make sure the vendor and product ID are the same as listed in the above lsusb output

5- Check the files for USB device:
ls -la /dev/ttyU*


Output will be similar to:

crw-rw---- 1 root uucp 188, 0 2009-08-16 19:36 /dev/ttyUSB0
crw-r--r-- 1 root root 188, 1 2009-08-16 19:34 /dev/ttyUSB1


if no USB0 or USB1 files listed, then try creating it as:
sudo mknod /dev/ttyUSB0 c 188 0
sudo mknod /dev/ttyUSB1 c 188 1


6- now hopefully your are done with the configuration part. just try connecting internet:
sudo wvdial

console will look like:
--> WvDial: Internet dialer version 1.60
--> Cannot get information for serial port.
--> Initializing modem.
--> Sending: ATZ
ATZ
OK
--> Sending: ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
OK
--> Modem initialized.
--> Sending: ATDT#777
--> Waiting for carrier.
ATDT#777
CONNECT
--> Carrier detected. Starting PPP immediately.
--> Starting pppd at Thu Aug 20 22:06:00 2009
--> Pid of pppd: 7268
--> Using interface ppp0
--> pppd: [10]�[17]
--> pppd: [10]�[17]


Congrats! your are connected!!! to DISCONNECT use ctrl+c.

Sunday, September 20, 2009

Few commands in Git source control system

I always prefer writing commands rather than using the tool and be a handicap. In Windows Explorer, right-click on the working directory you want and choose “Gui Bash Here”. Then enter a command like this:

To clone a repository from remote repository, you will have to type

git clone git@github.com:firstnamelastname/yourrepo.git

git clone repoURL

Git might prompt you about an SSH key, the first time you do this with github (or any other new server). Answer “yes”.

It’s worth pointing out here, if you didn’t already understand from the various Git web sites, that Git is a distributed source control system. It will pull down the whole project history, so you can browse history and even commit changes without online access. Thus Git works very well if you have an intermittent or poor network connection.

Few commands:

As with all source control, work in the directory where you use source control. Do not copy files back and forth between here and some other working directory, that is a path to endless merge and update problems.

Once you have checked out the software, here is a summary of your work flow. For more details, please read the copies Git documentation online. I suggest reading both the official Git material, as well as other sites and articles about Git.

Getting Changes

Get changes from others with “git pull” (or using the GUI). By default this will pull from the repo from which you cloned, so if you cloned the upstream repo, that will get other peoples’ changes.

If you cloned from your own Git hub repo, you’ll need to use something like this:

git remote add upstream git@github.com:firstnamelastname/yourrepo.git

git pull yourname upstream

Sending Changes

Commit your changes locally with “git commit” (or using the GUI). Remember that Git generally wants you to explicitly say which files’ changes to include (”git add”), so make sure you read and understand enough about Git to do this properly; it is only a few commands or clicks in the GUI. The usual caveat applies, to only commit actual source files, not generated files or temp files.

Push your changes up to your GitHub repository with “git push”. This step will make it so others on your project can see your changes. Do this at least once per day, and ideally more often as you collaborate. Assuming that you cloned from the upstream repo, you’ll need to set up a reference to your own Git hub repo (the one you can push to), with something like this:

git remote add name git@github.com:firstnamelastname/yourrepo.git

As usual, use reasonable names and relevant URLs, not my sample names and URLs. Once you’ve added the remote reference, pushing is easy:

git push name master

When you have a set of changes (one or more commits) that you think are ready to go in to the main-line of the project, use Git hub to issue a “pull request”.A key thing to understand about Git is that it makes branching extremely easy and fast, so that very convenient to use branches.

Installing Git version control from windows

This sentence confirms my ownership of the site and this site comply with the Terms and Conditions and program policies for Google AdSense. ca-pub-0819929743481189

  1. Go to http://code.google.com/p/msysgit/downloads/list and download the latest version of git and install it.
  2. Now open the Git bash and generate a ssh key. Type ssh-keygen -C "username@email.com" -t rsa ( When you click on 'Enter' key note where the ssh key is going to be saved"
  3. Now open the file where you have saved ssh key and note it down.
  4. Go to www.github.com
  5. Create your own git hub account. While creating the account you will be asked to enter the ssh key. Copy the saved ssh key.
  6. Now you need to create your own user name and email for the local repository. This will be information incase of multiple users committing the code to know who is the committing the code . Type this with your email and password
    git config –global user.email Your.Email@domain.com
    git config –global user.name “Your Real Name”
  7. Then you are ready to proceed with getting into a project. Copy the “Clone URL” from a github project page. Make a new directory on your machine, to become your working directory. There are two approaches to which project to clone.
  • Clone from your own fork repo. This will make it trivial to push your changes up, but require one more command to get upstream changes.
  • Clone from the upstream (my) repo. This will make it trivial to get change, but require one more command to be able to push changes, because you can’t push to another Github users’ repo.
Now if you have already created a repository in github.com, and want to clone it to your local machine then follow the below steps
  • Go to start --> All programs --> Git --> Git UI
  • A new window will open and click on "Clone Existing Repository"
  • Now it wil ask you to enter source location which will be something like git@github.com:firstnamelastname/yourrepository.git and target directory would be your rails_apps folder/yourappname
  • Click on 'clone'.
  • Incase you get "Permission denied" then type ssh git@github.com and press enter and then you should get a welcome message from git hub. If you have not received then you should do some more research on git.
  • Congratulations! you have successfully cloned a remote repository. Suggestions are welcome incase of errors in this blog.