When using mobile devices for a survey or census it is important to be able to transfer data collected in the field back to the head office. This allows for faster processing and analysis as well as better monitoring of the progress of the field operation.
In CSPro, transferring data between devices in the field and the head office is referred to as synchronization. CSPro supports data synchronization over the Internet between interviewers' devices in the field and a central server. Interviewers use CSEntry to collect data on tablets, phones, or laptops and then use the Internet to synchronize the data on their devices with a server at the head office or in the cloud. In CSPro, the data collection itself may be done entirely offline, with no Internet connection. When an interviewer is able to connect to the Internet they may synchronize with the server and transfer any data collected since the last synchronization. Synchronization may be done using Wi-Fi or a mobile data connection (2G/3G/4G).
Direct synchronization between interviewers and central server over the Internet
In situations where an interviewer rarely or never has access to the Internet, CSPro supports peer-to-peer synchronization between devices using Bluetooth. Bluetooth synchronization does not require an Internet connection. It is a direct connection between two devices that are in close proximity. Using Bluetooth, an interviewer may synchronize their device with a supervisor's device, transferring their data to the supervisor. Later, the supervisor travels to a location where they are able to connect their device to the Internet in order to synchronize with the central server. In this scenario, a supervisor might visit multiple interviewers in order to synchronize with them and later upload their data to the server at the head office.
Synchronization with supervisor over Bluetooth when interviewers do not have Internet access
CSPro supports both one-way and two-way data synchronization. This means that is possible to send data from the interviewer's device to the server as well as to download data from the server to the interviewer's device. You can configure your synchronization to only send data to the server, to only receive data from the server, or to both send and receive data.
It is possible to synchronize both CSPro data files and other non-data files such as application files, images, and text files. It is possible to update data entry applications in the field by downloading the latest .pen
files from the server. This way, modifications to the application at the head office can easily be distributed to interviewers in the field. It is, however, simpler to deploy and update application files on mobile devices using the Deploy Application
tool. This tool packages application files for deployment, uploads them to a server and allows interviewers in the field to install and update them on a tablet by choosing Add Application
from the menu on their mobile device.
For synchronization over the Internet, a central server is required. CSPro supports three types of servers: CSWeb, Dropbox, and FTP.
- CSWeb: CSWeb is a web server running the CSPro synchronization server software. It is best for very large surveys and censuses. The server software is written in PHP and can be run on any web server that has PHP and the MySQL database software installed. The server can be set up on a computer at the head office that is connected to the Internet or it may be set up on a hosted website or virtual server in the cloud. Setting and maintaining a server for CSWeb requires knowledge and experience with web server maintainence and cyber security. Users that do not have these skills should consider using Dropbox or FTP instead. For more information about CSWeb servers, see the CSWeb help documentation.
- Dropbox: Dropbox is a free cloud based synchronization service. It is ideal for those doing small and medium survey operations. Dropbox requires no server setup or maintenance and avoids the cost and difficulty of setting up a CSWeb server.
- FTP: CSPro synchronization can be used with a FTP (file transfer protocol) server. For those doing small and medium survey operations who do not want to use Dropbox and are not able to configure a server to use CSWeb, FTP can be used for synchronization.
When synchronizing CSPro data files, synchronization is done at the case (questionnaire) level. CSPro keeps track of which cases have been added or updated and only sends cases that are new or have been modified since the last synchronization. This significantly reduces the amount of data transferred and therefore reduces bandwidth and the cost of air time. It also reduces the chance that two interviewers overwrite each other's work by both syncing to the same data file on the server. As long as two interviewers do not modify the same case at the same time, they may both synchronize to the same server without overwriting each other's data.
Once interviewers in the field have used data synchronization to upload data to the server, you can use the Data Viewer tool to download the data. When using CSWeb for data file synchronization, data is stored in a MySQL database on the server. When using either Dropbox or FTP, data will be stored on the server in a set of files in the directory CSPro/DataSync
. In both cases, the data on the server cannot be opened directly by CSPro. In order to retrieve the data that has been uploaded to the server, use the Data Viewer's download function
. This will download all the data on the server into a single CSPro data file that can be used by other CSPro applications and tools.
Basic synchronizations can be set up by specifying a few options documented here
For more advanced synchronization scenarios, CSPro has logic functions that can be used to implement data and file synchronizations. Using logic, it is possible to synchronize data files, including those associated with external dictionaries, and non-data files over either the Internet or Bluetooth.
There are several logic functions used for synchronization with a server:
- syncconnect is used to connect to the server and initiate the synchronization session.
- syncdisconnect is used after synchronizing to end the session.
- syncdata is used to synchronize cases in CSPro data files with the server.
- syncfile is used to upload/download non-data files to/from the server.
- syncparadata is used to synchronize paradata to/from the server.
- syncapp is used to update an application on the device from a new version on the server.
- syncmessage is used to send to and receive string messages from a Bluetooth server.
While it is possible to use syncfile
to synchronize CSPro data files, it is more efficient to use syncdata
since it will only send/receive cases that have been modified whereas syncfile
always sends the entire file.
You will generally call syncconnect
once to start the session followed by multiple calls to syncdata
, and syncfile
, followed by a call to syncdisconnect
. In the example below, we connect to the server, download the latest application files and upload cases from the data file associated with the data dictionary SURVEY_DICT
Bluetooth synchronization is done between two devices. One device, called the client, has logic similar to what is used for web synchronization. The second device plays the role of server. With Bluetooth synchronization, the server is passive. The device acting as the server runs only a single logic function, syncserver
, which waits and processes commands from the client device. Only the client device runs syncconnect
, and syncmessage
The client and server devices must be in close physical proximity in order to connect to each other. The server device must first execute the syncserver
function which causes it to wait for connections from nearby devices. Then the client device executes syncconnect
which initiates the connection between the two devices. The client then executes one or more calls to syncdata
and/or other sync functions, followed by a call to syncdisconnect
, which ends the session. This requires that the two operators using the devices coordinate to start the synchronization routines on the two devices at roughly the same time.
The following example shows the logic that would be written for the client and for the server:
// start peer-to-peer Bluetooth server
The synchronization functions all assume that your CSPro application is already installed on the interviewers device. This means that it is not possible to use the above methods to download a new application for initial installation. While it is possible to simply copy the application to the interviewers device by connecting it to a PC via USB, in some cases it is preferable for the interviewer to download applications over the Internet. To download an application onto a device, use Add Application
from the menu in the Entry Applications
screen in CSEntry on an interviewer's mobile device. To package and upload an application to a server from where it can be downloaded, use the Deploy Application
tool. In addition, once an application has been installed on a mobile device using the Deploy Application
tool, it can easily be updated by re-deploying the application to the server and then using Update Installed Applications
from the menu on the mobile device or by using syncapp
in application logic.
Many errors that you encounter during synchronization are caused by lack of network connectivity. An easy way to check if an error is a problem with your CSPro program rather than a general network error is to try to connect to the sync server in the web browser on the device you are trying to synchronize. For example, if you are trying to synchronize data on a tablet using Dropbox, try to connect to https://www.dropbox.com in the web browser on your tablet. If you are unable to connect in the web browser then there is a problem with your connection to the Internet rather than a problem with your CSPro application.
To get additional details on synchronization errors you can look at the sync.log file. This files lists all synchronization oprtations along with any errors encountered. This includes errors in uploading/downloading applications in Application Deployment and data download in Data Viewer. On Android devices you can find the sync.log file in the CSEntry directory on your tablet or phone. On Windows you can find the file in the directory %AppData%\CSPro. On Windows you can also locate the sync.log file from the Help menu in CSPro Designer under Troubleshooting. When asking for technical support with synchronization issues, please attach the sync.log file from your device to your email or forum post.