Data Viewer - Download or Synchronization

[khurshid.arshad]

Dear CSPro Team;

I am using CSPro 7.4 with CSweb 7.3.1. I have 129,900 cases in my listing diction on the server.

Before using 7.4 version, i was using 7.1.3 CSEntry with CSweb 7.1.2. Now, I made a new database in PhpMyAdmin and transfer previous data in to this. Now my field teams are sending data in my new database.

When i try to download data from the server it shows only 126,007 out of 129,900 cases.

The Procedure which i am following:

• When i use File-> download (with new file name), i get only 1496 out of 129,900 cases in my new data file.

• When i use File-> synchronization (an existing file), it shows only 126,007 out of 129,900 cases.

Please advice.

Best regards.
a.

[savy]

Can you send the sync.log file from the machine that is downloading the data? It is located in %appdata%\CSPro directory
In the case where you have existing csdb does the total count of cases include deleted cases if any?

[khurshid.arshad]

Dear Savy;

I hope you are doing well.

Please see attached file.

Best.
a.

[savy]

Dear Arshad,

I do not see any problem with the sync. How did you transfer the old data to the new db?

Thanks

Savy

[khurshid.arshad]

Dear Savy;

Thank you for your prompt reply. This error was due to duplication in my data set.

Now it is fixed.

Thanks.
a.

[khurshid.arshad]

Dear Savy;

One more thing about data downloading. There are two options to data downloading, one is Sychronization with existing data and the other one is downloading only new cases or change cases.

However i want to download all data again, changed or unchanged.

How should i go about it.

Best.

a.

[savy]

You will be able to download all the cases only when it is a new file. That is the easiest option for you.

Thanks

Savy

[khurshid.arshad]

Dear Savy;

In my early mail, I have mentioned that I was using 7.1 for my current project. Now I am using 7.4 with 7.3 CSWeb for this project.

I have followed the following steps to import data from one database to another to use in csweb7.3

Under PhpMyAdmin -> export database and -> save on local drive with sql extension and then import in another database within same PhpMyAdim.

Now i have 130,908 cases including previous cases. Carry forward cases are 126,048.

As you have mentioned that we can download all cases when it is a new file.

Does this mean that we cannot download all cases again?

Thanks.
a

[josh]

In Data Viewer if you use File->Download and specify a new csdb file it will download all the cases. If you choose Synchronize or File->Download and use an existing csdb file it will download only the cases that have not already been downloaded to the csdb file you chose. In either case the end result should be that the csdb now has all the cases that are on the server. The difference is that if you have a lot of cases on the server it is faster to use an existing file and just add the new/modified cases to it rather than downloading everything again. But in both scenarios you should end up with the same cases in the local csdb file and on the server.

The way this works is that in the csdb file, CSPro saves the last version of the database when you last downloaded/synchronized from the server in the csdb file. When you synchronize/download with an existing file it asks the server to send only the cases that are new or have been modified since that last version. This way only data that have changed or are new are downloaded. When you use a new empty csdb file there is no previous revision so it downloads everything. If you look in the sync.log file you can see this. If you are using an empty you will see something like First time GET with this server meaning that there is no previous download/sync so it will ask the server to send all the cases. If you use an existing file you will see a line like Last GET with this server 2020-06-21 12:50:56 local rev 2 server rev "22" which indicates when the last download/sync was and the version of the database at that time. In either case at the end of the sync/download you will see a line like 5 new cases, 2 updated. 0 ignored, 0 conflicts. The first number is the number new cases that were added to the server since the last download and were downloaded to the csdb file. The second is the number of existing cases that had previously been downloaded but were then modified and were therefore downloaded again. The third and fourth numbers are cases that were modified on both the server and in the local csdb file since the last sync. If you are just downloading data and not making changes locally then these should be both be zero. If you are changing the file locally and interviewers in the field are also modifying the same cases and uploading them to the server then you will get conflicts.

[khurshid.arshad]

Dear Josh

Steps which i am following to download all the 135004 cases:

CSPro 7.4 -> Tools -> DataViewer ->File -> Download -> CSWeb Server -> http:Servername -> connect->

Save as: New Data.csdb

After that i am getting only 6610 out of 135004 cases.

Synchronization information with new file.

2020-06-23 14:16:42,601 INFO: Connect to server http://ServerName using saved credentials
2020-06-23 14:16:42,601 INFO: Connect to server http://ServerName
2020-06-23 14:16:42,763 INFO: Server API version: 1.1
2020-06-23 14:16:42,763 INFO: Connection successful. Server id: http://localhost/csweb7_3/api/
2020-06-23 14:16:42,780 INFO: Downloading dictionary list
2020-06-23 14:16:43,256 INFO: Downloading dictionary list complete
2020-06-23 14:17:20,447 INFO: Downloading dictionary file: LISTING_DICT_NSER
2020-06-23 14:17:20,575 INFO: Downloading dictionary file complete
2020-06-23 14:17:20,632 INFO: Syncing data: New Data.csdb direction GET universe ""
2020-06-23 14:17:20,632 INFO: First time GET with this server
2020-06-23 14:17:20,633 INFO: Start case download with chunk size of 100
2020-06-23 14:17:21,059 INFO: Status : 206
2020-06-23 14:17:21,071 INFO: Cases in chunk 100. Server count = 100/6610
2020-06-23 14:17:21,071 INFO: Chunk time Seconds : 0.425575 Cases: 100 Kb: 175.726 mbps 3.30331
2020-06-23 14:17:21,071 INFO: Update chunk size to 200 cases
2020-06-23 14:17:21,097 INFO: Start case download with chunk size of 200
2020-06-23 14:17:21,097 INFO: Download chunk starting from case 4a094124-b182-4508-ad3d-eca3b5aa9be9
2020-06-23 14:17:21,355 INFO: Status : 206
2020-06-23 14:17:21,375 INFO: Cases in chunk 200. Server count = 200/6510
2020-06-23 14:17:21,376 INFO: Chunk time Seconds : 0.257598 Cases: 200 Kb: 348.797 mbps 10.8323
2020-06-23 14:17:21,376 INFO: Update chunk size to 400 cases
2020-06-23 14:17:21,410 INFO: Start case download with chunk size of 400
2020-06-23 14:17:21,410 INFO: Download chunk starting from case decebd69-dc22-42fd-bd00-2e01d31f477a
2020-06-23 14:17:21,758 INFO: Status : 206
2020-06-23 14:17:21,790 INFO: Cases in chunk 400. Server count = 400/6310
2020-06-23 14:17:21,790 INFO: Chunk time Seconds : 0.347952 Cases: 400 Kb: 698.31 mbps 16.0553
2020-06-23 14:17:21,790 INFO: Update chunk size to 800 cases
2020-06-23 14:17:21,826 INFO: Start case download with chunk size of 800
2020-06-23 14:17:21,826 INFO: Download chunk starting from case 190b9489-bbb6-4cae-a095-858d110bb2e3
2020-06-23 14:17:22,767 INFO: Status : 206
2020-06-23 14:17:22,821 INFO: Cases in chunk 800. Server count = 800/5910
2020-06-23 14:17:22,821 INFO: Chunk time Seconds : 0.94134 Cases: 800 Kb: 1391 mbps 11.8214
2020-06-23 14:17:22,821 INFO: Update chunk size to 1600 cases
2020-06-23 14:17:22,893 INFO: Start case download with chunk size of 1600
2020-06-23 14:17:22,893 INFO: Download chunk starting from case 52480668-0d96-427b-a4ab-f0aacbc04b9b
2020-06-23 14:17:24,314 INFO: Status : 206
2020-06-23 14:17:24,404 INFO: Cases in chunk 1600. Server count = 1600/5110
2020-06-23 14:17:24,404 INFO: Chunk time Seconds : 1.42055 Cases: 1600 Kb: 2731.47 mbps 15.3826
2020-06-23 14:17:24,404 INFO: Update chunk size to 3200 cases
2020-06-23 14:17:24,534 INFO: Start case download with chunk size of 3200
2020-06-23 14:17:24,534 INFO: Download chunk starting from case d78c250e-2c5f-4d47-ab77-0aae52dfe5ed
2020-06-23 14:17:27,565 INFO: Status : 206
2020-06-23 14:17:27,702 INFO: Cases in chunk 3200. Server count = 3200/3510
2020-06-23 14:17:27,702 INFO: Chunk time Seconds : 3.02976 Cases: 3200 Kb: 5354.49 mbps 14.1384
2020-06-23 14:17:27,702 INFO: Update chunk size to 6400 cases
2020-06-23 14:17:27,994 INFO: Start case download with chunk size of 6400
2020-06-23 14:17:27,994 INFO: Download chunk starting from case 93d843fe-f809-41e3-b62d-6a0bcb50a64a
2020-06-23 14:17:28,232 INFO: Status : 200
2020-06-23 14:17:28,262 INFO: Cases in chunk 310. Server count = 310/310
2020-06-23 14:17:28,328 INFO: New server revision = 183
2020-06-23 14:17:28,328 INFO: Sync GET completed.
2020-06-23 14:17:28,328 INFO: Downloaded 6610 cases
2020-06-23 14:17:28,328 INFO: 6610 new cases, 0 updated, 0 ignored, 0 conflicts
2020-06-23 14:17:28,329 INFO: Disconnecting from server



thanks.
a.