Dictionary merging

Discussions about editing and cleaning data
Forum rules
New release: CSPro 8.0
Post Reply
Sol

Dictionary merging

Post by Sol »

Hi All,
I have two dictionaries with same ID field i.e they can be linked with this ID. Please guide me as how to merge the two dictionaries in to one dictionary file. How can I also merge the data files associated with these dictionaries?
Thanks
Gregory Martin
Posts: 1777
Joined: December 5th, 2011, 11:27 pm
Location: Washington, DC

Re: Dictionary merging

Post by Gregory Martin »

If you're merging records from different dictionaries, you first must make sure that the ID field is in the same position in both dictionaries and that none of the record types in each dictionary are the same. If this is the case, then I would:

1) Create a new dictionary, NewDict.dcf, by creating a copy of A.dcf.

2) Copy all of the records from dictionary B.dcf and paste them into dictionary NewDict.dcf. You an do this quickly by selecting all of the records (from the dictionary level view), copying them to the clipboard, and then pasting them on the level view of NewDict.dcf.

Now you should have a dictionary that describes the records from dictionaries A.dcf and B.dcf. Next you want to merge the data files associated with the dictionaries. For this you use the Concatenate Data tool to create a new file with the records from both data files.

Finally you need to sort this new data file so that the records from the two files are in order by ID. For this you use the Sort Data tool, selecting as the sort type "Record Sort (ids and record type only)." Add all of the IDs to the sort query, and then add the record type as the last value to sort on. Run this program, and now you have a data file that NewDict.dcf can read.

Let me know if you have questions.
Sol

Re: Dictionary merging

Post by Sol »

Thanks martine,

Your response is very helpful. The sort program responded with error since it considered the ids from the second data file were duplicates of the first data file. We have behavioral and biological data for one respondent but with separate dictionnaries with the same id, we were assuming merging the two data files for analysis will be easy.

Thanks again
Gregory Martin
Posts: 1777
Joined: December 5th, 2011, 11:27 pm
Location: Washington, DC

Re: Dictionary merging

Post by Gregory Martin »

Are you sure that you did a record sort? When you select the option I highlighted, CSPro will join the cases together before checking if there are duplicates in your file.
Sol

Re: Dictionary merging

Post by Sol »

Sorry Martin, I am late.

Thanks so much. It worked well
Amandine

Re: Dictionary merging

Post by Amandine »

Dear all,

I've followed your example for the dictionaries and it works well ! But I've a problem with the sort data tool.

In fact, I would like to compare results from two different survey with exactly the same IDs. Do we need two compare survey to have the same number of case? And if yes, how can I delete the surplus case ?
When I run the sort data tool, I've an error message as follow : "Sort Completed. Structure errors in resulting file"...

Thanks for your help,

Amandine
Gregory Martin
Posts: 1777
Joined: December 5th, 2011, 11:27 pm
Location: Washington, DC

Re: Dictionary merging

Post by Gregory Martin »

I sent you an email about this. Did it address your concerns?
munirmdee1
Posts: 74
Joined: August 17th, 2015, 9:32 am
Location: Dar es Salaam, Tanzania

Re: Dictionary merging

Post by munirmdee1 »

Hi, Gregory. I have the same issue over here, but a little bit different. I have three dictionaries, Household1,Household2 and Individual. In Household1 and Household2 the ID is the same which is only HouseholdNumber, but in Individual ID is made up with two variables which are HouseholdNumber and IndividualID. In Household1 there are section A to B, and in Individual there are section C to E, and in Household2 there are sections F to J. How can I merge all these three dictionaries together and their datafiles?
Munir Mdee
Software Programmer
National Bureau of Statistic (NBS)
Jakaya Kikwete Road,
P.O.Box 2683,
Dodoma,TANZANIA
Mob: +255 755 740090
Email: munir.mdee@nbs.go.tz
munirmdee@gmail.com
josh
Posts: 2399
Joined: May 5th, 2014, 12:49 pm
Location: Washington DC

Re: Dictionary merging

Post by josh »

For the individual sections I would first convert those from single occurrence records to multiplying occurring records and make the household id be the only id-item. The individual-id would just become a regular variable on each of the individual records. Make sure to keep the start positions and lengths the same. Then you can merge the individual and household records together as Greg describes.
Dawazangpo
Posts: 3
Joined: February 2nd, 2016, 4:49 am

Re: Dictionary merging

Post by Dawazangpo »

Hi martine and gregory,
I have same issue and I did what you have explained in above queries and I have sorted the input files i.e data files after concatenating and when I try to export the sorted file. it says no output generated.......file not created. What could be the problem now.
Post Reply