Problem concatenate several files
Problem concatenate several files
When I concatenate several files it repeats the header of the variables of any files in the file end using Cocatenate Tool. How can I solve this problem?
-
- Posts: 1801
- Joined: December 5th, 2011, 11:27 pm
- Location: Washington, DC
Re: Problem concatenate several files
The concatenate tool was designed for combining CSPro data files, which don't contain headers, so the situation that you are faced with is slightly unusual. You could combine multiple files using a batch application, only writing out the header once, but this might be a lot of work for what you are trying to accomplish. See the attached application, and look at this logic:
PROC GLOBAL
numeric numFiles = 2;
array alpha (100) filenames(numFiles) = "file1.txt"
"file2.txt";
alpha (100) outputFilename = "output.txt";
alpha (300) str;
file inFile,outFile;
PROC NOREPEATEDHEADERS_FF
preproc
setfile(outFile,outputFilename,create);
numeric i;
do i = 1 while i <= numFiles
if setfile(inFile,filenames(i)) then
if fileread(inFile,str) and i = 1 then // write out the header for only the first file
filewrite(outFile,strip(str));
endif;
while fileread(inFile,str) do
filewrite(outFile,strip(str));
enddo;
close(inFile);
endif;
enddo;
close(outFile);
numeric numFiles = 2;
array alpha (100) filenames(numFiles) = "file1.txt"
"file2.txt";
alpha (100) outputFilename = "output.txt";
alpha (300) str;
file inFile,outFile;
PROC NOREPEATEDHEADERS_FF
preproc
setfile(outFile,outputFilename,create);
numeric i;
do i = 1 while i <= numFiles
if setfile(inFile,filenames(i)) then
if fileread(inFile,str) and i = 1 then // write out the header for only the first file
filewrite(outFile,strip(str));
endif;
while fileread(inFile,str) do
filewrite(outFile,strip(str));
enddo;
close(inFile);
endif;
enddo;
close(outFile);
- Attachments
-
- noRepeatedHeaders.zip
- (2.82 KiB) Downloaded 567 times
Re: Problem concatenate several files
Could you please share an example of a batch file to concatenate several data files if you have.
I would be very interested.
Thank you
I would be very interested.
Thank you
-
- Posts: 1801
- Joined: December 5th, 2011, 11:27 pm
- Location: Washington, DC
Re: Problem concatenate several files
The example above is a program that concatenates several files, only writing out the first line of a file once.
What kind of example are you looking for?
What kind of example are you looking for?
-
- Posts: 67
- Joined: June 26th, 2017, 2:15 pm
Re: Problem concatenate several files
I have two separate data files in .csdb format. when i concatenate these files, the cspro concatenate tool says :
Concatenation done with the following errors:
Skipping duplicate case 740203181922 in file C:\UPMPR17\inputdata\achdata\मासिक_प्रगति_मार्च_1819_मुरादाबाद मण्डल_09_08_2019_17_33_20.csdb. Previously found in file C:\UPMPR17\inputdata\achdata\मासिक_प्रगत
Skipping duplicate case 740203181923 in file C:\UPMPR17\inputdata\achdata\मासिक_प्रगति_मार्च_1819_मुरादाबाद मण्डल_09_08_2019_17_33_20.csdb. Previously found in file C:\UPMPR17\inputdata\achdata\मासिक_प्रगत
but both files have distinct id.
Please tell the reason and guide me how to resolve it.
Concatenation done with the following errors:
Skipping duplicate case 740203181922 in file C:\UPMPR17\inputdata\achdata\मासिक_प्रगति_मार्च_1819_मुरादाबाद मण्डल_09_08_2019_17_33_20.csdb. Previously found in file C:\UPMPR17\inputdata\achdata\मासिक_प्रगत
Skipping duplicate case 740203181923 in file C:\UPMPR17\inputdata\achdata\मासिक_प्रगति_मार्च_1819_मुरादाबाद मण्डल_09_08_2019_17_33_20.csdb. Previously found in file C:\UPMPR17\inputdata\achdata\मासिक_प्रगत
but both files have distinct id.
Please tell the reason and guide me how to resolve it.
- Attachments
-
- merge_achdata.zip
- concatenate app
- (2.47 KiB) Downloaded 236 times
Re: Problem concatenate several files
It is hard to say exactly what is going since your attachment does not contain the data files themselves. You will get the message about duplicate cases in two situations:
1) if between the two files you have two cases with the same case ids i.e. the same values for all of the id variables.
2) if between the two files you have two cases with the same internal id. This occurs if you created the case on one device and sent it another device either by sync or copying the file via USB and then modified the id variables on the second device. When you modify the id variables the internal id still the same so the concat tool treats it as a duplicate case.
1) if between the two files you have two cases with the same case ids i.e. the same values for all of the id variables.
2) if between the two files you have two cases with the same internal id. This occurs if you created the case on one device and sent it another device either by sync or copying the file via USB and then modified the id variables on the second device. When you modify the id variables the internal id still the same so the concat tool treats it as a duplicate case.
-
- Posts: 67
- Joined: June 26th, 2017, 2:15 pm
Re: Problem concatenate several files
Thanks for response.Here are the two data files one of April (23 cases) and another of May(23 cases) but when concatenated is shows only 23 cases saying skipping duplicate cases id.But both files have distinct ids.Please guide me how to concatenate cases in this situation..Can i change myself internal case ids.
- Attachments
-
- datafiles.zip
- (27.66 KiB) Downloaded 217 times
Re: Problem concatenate several files
The internal ids of the cases in the two files are all the same. I'm guessing you copied the data file for April and then modified the cases to create the file for May rather than creating a new file. To remove the internal ids you can convert the files to text format, which has no internal ids, and then concatenate the text files. If you want a csdb file in the end you can convert the concatenated file back to csdb although depending on what you want to do with the concatenated file that may not be needed.
-
- Posts: 67
- Joined: June 26th, 2017, 2:15 pm
Re: Problem concatenate several files
Thanks a lot..
-
- Posts: 67
- Joined: June 26th, 2017, 2:15 pm
Re: Problem concatenate several files
One more thing i want to do is to compare the data of a month from the previous month.I have created the pff application to generate report in html.For Example i want to display a column value of both the current month and the previous month at a time.For example based on the earlier attached data file i want to display a column value for May and April month side by side. Image of report is uploaded. Please help me to do the required.
- Attachments
-
- ach_image.jpg (81.4 KiB) Viewed 5490 times