Reformat not required records

Discussions about tools to complement CSPro data processing
Post Reply
AriSilva
Posts: 480
Joined: July 22nd, 2016, 3:55 pm

Reformat not required records

Post by AriSilva » July 17th, 2021, 11:23 am

I´m reformatting a dictionary changing the id.
In the source dictionary the id is the geography plus a house id, and it has one case for each hhold, and zero or more records for the persons (that is, the person record is not required). The person record has a unique person ID, which I´m moving to the record id of the target dictionary.
The source id is moved to the person´s record in the target dictionary.
So, the idea is to have an output file with one record per person, with no data for the hhold.
Everything worked smoothly, but for some blank records that, I presume, belong to the cases where there were no persons in the hhold.
Of course I can write an utility program to get rid of these blank records.
Is there a way to filter the non existent persons in the source file (maybe it should be a new feature for the reformat tool), in order for them not to generate blank records?
Best
Ari
Best
Ari

AriSilva
Posts: 480
Joined: July 22nd, 2016, 3:55 pm

Re: Reformat not required records

Post by AriSilva » July 17th, 2021, 12:29 pm

Sorry, the export is not working well, it is exporting only the first person of the household.
What should I have to do to export all the persons in the household?
Best
Ari

sherrell
Posts: 28
Joined: April 2nd, 2014, 9:16 pm
Location: Washington, DC

Re: Reformat not required records

Post by sherrell » July 19th, 2021, 12:46 am

Hi Ari,

To confirm what you're doing, you're using the reformat utility or the export utility?

And you have two records, a required the housing record and 0+ person records, but you only want the non-blank person records, yes?

I would think this would work fine with the export utility, as long as you choose one of the person variables that you know won't be blank/empty as a universe. Could you share the dictionary and a few dummy records?

Thanks,
Sherrell

AriSilva
Posts: 480
Joined: July 22nd, 2016, 3:55 pm

Re: Reformat not required records

Post by AriSilva » July 19th, 2021, 6:09 pm

Thanks, Sherrell, for your very prompt answer.
I´ve thought about using the export instead of the reformat, but it would not work since the export needs at least one id variable to be exported, and my target file is based on an ID variable that is inside the person record.
I´m sending you an example of what I need.
1. Input file - ELSI_40_Minimalista.csdb it has only 30 records
It uses the ELSI_14_Entrevista_Minimalista.dcf dictionary
I´m focusing on record B_MORADORES_ANTIGOS, which is not required and has up to 5 records
This example file has 10 records with no B_MORADORES record, 10 records with just one "morador", and 10 records with 2 "moradores".

2. The reformated dictionary
ELSI_20_Idoso_Antigo_Example.dcf
If you notice, the ID is PPO71, which is the identification of the person in the B_MORADORES_ANTIGOS record.
I´ve forced the output record as nor required with just one record per case id
The old ID was moved inside the B_MORADORES_ANTIGOS record

3. The reformat pff
ELSI_21_Idoso_Antigo_Reformat.pff

4. Output file (or what I would expect)
ELSI_20_Idoso_Antigo_Output.csdb
This file should have 30 persons, and it has 30 records, but 10 of them are empty, corresponding to the 10 cases where there was no B_MORADORES_ANTIGOS record.
And the cases where there was 2 B_MORADORES_ANTIGOS, just the first one was reformated to the output.

I hope you understand what I´m trying to do with the reformat.
If it does not work I will write a batch program to write on output .dat file, but first I would create some group items to make my life easier when writing the variables.
All the best
Attachments
reformat.rar
(49.89 KiB) Downloaded 6 times
Best
Ari

AriSilva
Posts: 480
Joined: July 22nd, 2016, 3:55 pm

Re: Reformat not required records

Post by AriSilva » July 20th, 2021, 10:07 am

You´ve given me a very good idea about exporting the file.
It does not do the job, but I can use it as a first step to get what I need:
1. Export to cspro just the person record I need. This will get rid of the questionnaires with 0 persons.
2. Reformat this auxiliary cspro file to the target dictionary, changing the questionnaire geographical id to the person id I want.
And it worked!
Thanks again
Best
Ari

sherrell
Posts: 28
Joined: April 2nd, 2014, 9:16 pm
Location: Washington, DC

Re: Reformat not required records

Post by sherrell » July 20th, 2021, 3:11 pm

Hi Ari

Yes, the reformat doesn't work quite as well as the export in your situation. Another option after the export is to just run a notepad++ macro (or equiv in another ASCII editor) to strip out whatever ID you force-selected for the export. Anyway, glad you got it to work w/o too much headache!

Sherrell

Post Reply