error 500 when synchronizing

Discussions about syncing data via Bluetooth, Dropbox, FTP, and using CSWeb
AriSilva
Posts: 383
Joined: July 22nd, 2016, 3:55 pm

Re: error 500 when synchronizing

Post by AriSilva » August 1st, 2020, 8:22 am

By the way, by looking at the sync_history, what are the fields file_revision, time_stamp and server_revision?
How can I convert the time_stamp to an actual date?
Best
Ari

josh
Posts: 2186
Joined: May 5th, 2014, 12:49 pm
Location: Washington DC

Re: error 500 when synchronizing

Post by josh » August 1st, 2020, 8:48 am

Ari - I feel like we had this problem where some cases were not getting uploaded before with a different one of your surveys but I can't remember if or how you solved it.

Timestamp is a standard unix timestamp - seconds since Jan 1, 1970. This website converts back and forth https://www.epochconverter.com. You can also convert it using sqlite queries: https://stackoverflow.com/questions/343 ... formatting or using CSPro logic https://www.csprousers.org/help/CSPro/t ... ction.html.

File revision is a link to the id column of the file_revisions table in the csdb. Each case in the cases table also has a file revision number (last_modified_revision) which links to the id of the file_revisions table and indicates when that case was last added/modified. When uploading cases we upload only those cases who's last_modified_revision is greater than the file_revision from the last sync_history entry with the same server. So what Aaron is saying is that if you delete all the rows in the sync history table in a csdb file the next time you upload it will upload all the cases instead of just the new/modified cases.

Server revision is the revision number from the sync_history table in the CSWeb database. When we upload cases to the server the server adds a new entry into its sync_history table and sends back the id of that new row. When uploading the next time we send that server revision to the server and it checks to make sure it has that revision in it's sync_history table. If it does not have it then it tells CSPro to upload all the cases instead of just the new/modified cases. This means that you if you clear the sync_history table on the server the next time you upload from any device it will upload all the cases.

AriSilva
Posts: 383
Joined: July 22nd, 2016, 3:55 pm

Re: error 500 when synchronizing

Post by AriSilva » August 2nd, 2020, 7:42 am

Hi Josh,
I do not remember having this kind of a problem before with one of our surveys.
I mean, not updating the server, from a client.
Anyway, as you explained on your note, I could delete all the sync_history data in the server to be able to force everybody to upload everything the next time they do a put, is that correct?
Is it safe to do so?
Can I do it anytime, or I should wait for a period of time when nobody is accessing the server? (at night for example).
Best
Ari

AriSilva
Posts: 383
Joined: July 22nd, 2016, 3:55 pm

Re: error 500 when synchronizing

Post by AriSilva » August 2nd, 2020, 8:44 am

Instead of deleting the sync_history from the client database, I´ve tried to simulate it by executing a cspro batch program to read/write the file, generating an output file. As I saw it in the SQLite, the sync_history of this output file is empty, of course.
Then I tried to upload this output file into the server, and, I do not know why, it did not update the server!
Even with the empty sync_history!
What am I doing wrong?
Best
Ari

josh
Posts: 2186
Joined: May 5th, 2014, 12:49 pm
Location: Washington DC

Re: error 500 when synchronizing

Post by josh » August 2nd, 2020, 9:02 am

I was thinking of this exchange viewtopic.php?f=11&t=3993&p=12418#p12418 but I don't see any resolution.

Are there cases in the csdb file that are not already on the server based on the case UUID? If you the cases in the csdb are already on the server (server has a case with same UUID) then the ones in the csdb may not overwrite the ones on the server. When you look at the sync.log on the client how many cases does it say it uploaded?

AriSilva
Posts: 383
Joined: July 22nd, 2016, 3:55 pm

Re: error 500 when synchronizing

Post by AriSilva » August 2nd, 2020, 2:27 pm

Follows the sync.log data
2020-08-02 15:15:22,140 INFO: Connect to server http://www.2clv.com.br/csweb74/api/ using saved credentials
2020-08-02 15:15:22,140 INFO: Connect to server http://www.2clv.com.br/csweb74/api/
2020-08-02 15:15:22,760 INFO: Server API version: 1.1
2020-08-02 15:15:22,761 INFO: Connection successful. Server id: 852995f1-c6b3-4bae-8f1a-14952401c8e5
2020-08-02 15:15:22,867 INFO: Syncing data: COVID_ELSI2_10_Entrevista.csdb direction PUT universe ""
2020-08-02 15:15:22,867 INFO: First time PUT with this server
2020-08-02 15:15:22,868 INFO: Total new/modified cases since last sync: 2
2020-08-02 15:15:23,352 INFO: Chunk time Seconds : 0.481644 Cases: 2 Kb: 6.01 mbps 0.0998248
2020-08-02 15:15:23,352 INFO: Uploaded chunk of 2 cases
2020-08-02 15:15:23,548 INFO: New server revision = 27172
2020-08-02 15:15:23,548 INFO: Sync PUT completed.
2020-08-02 15:15:23,548 INFO: Uploaded 2 cases
2020-08-02 15:15:23,688 INFO: Disconnecting from server

My test file has just 2 cases.
I look at the server data after downloading it and the data in the server is the old one, the change is not reflected there.
The uuids are the same in the test case and the server.
What you are saying is that if the uuids are the same the software does not update the server?
Even if you change the data in the client for that specific case, it is not updated in the server?
I always understood that if data is changed in the client the server is updated accordingly.
How come we always worked with that kind of assumption and it never happened before?
Did you change this behavior from 7.3 to 7.4?
What do we have to do to update the data in te server?

Do you mind to have a look at my files in the server and in the test case?
What should I send you?
a. the server´s direction (in the sync.log)
b. username: arisilva and psw: 7579nrNR
test file attached covid_elsi2_10_entrevista.csdb
case id: 00172

to see any change look at field w12_observacao, the server has "ari - nao atualiza o servidor"
the test data has nothing in this field. w10_observacao is also different in the server and the test
Attachments
COVID_ELSI2_10_Entrevista.zip
(15.35 KiB) Downloaded 2 times
Best
Ari

Post Reply