RESTful API calls to csweb server

Discussions about syncing data via Bluetooth, Dropbox, FTP, and using CSWeb
Post Reply
tercio
Posts: 3
Joined: March 18th, 2020, 4:51 am

RESTful API calls to csweb server

Post by tercio » March 18th, 2020, 5:14 am

Hi all,
I intend to develop a real-time dashboard that presents survey data on demand, therefore, we'd like to know if there is a way of making RESTful API calls to csweb endpoints, to retrieve data stored in the csweb mysql database. I want to create a frontend application that will make requests to the server (Using PHP or Angular, or other framework), and displays the data on the app.

Alternatively, I could take the data directly from the csweb database, but I could not understand the data in the mysql table... I could not see the survey variables in the database.

Please advise the best way to achieve this.

Regards,
Tércio

josh
Posts: 2062
Joined: May 5th, 2014, 12:49 pm
Location: Washington DC

Re: RESTful API calls to csweb server

Post by josh » March 18th, 2020, 7:29 am

CSWeb has a REST API. It is documented in csweb/src/api/app/swagger.json. That file is best using the tools here https://swagger.io/

The challenging part of what you want to do it is extracting the data from the CSPro text format that is stored in the database. You need to use the start positions and lengths of elements in the data dictionary to take substrings from the data and convert them to numbers. There are a couple of open source projects that have already done this that you should look at:

https://github.com/IstatCooperation/CSProDashboard
https://medium.com/@nahomt/working-with ... 6161b84ffa

htuser
Posts: 456
Joined: December 19th, 2011, 6:26 pm
Location: HAITI

Re: RESTful API calls to csweb server

Post by htuser » March 18th, 2020, 1:42 pm

Dear Tercio and Josh,
I intend also to use best in class dashboard https://superset.apache.org/ for live reporting of:
a) Census/Survey data;
b) Geo-enableb monitoring and evaluation system;
c) or whatever information system implemented using CSPro;

But, always waiting automatic data parsing on CSWeb (relational schema support as requested long ago, and also by Tercio);
I already tested CSpro2SQL, but it's limited...
I've just ask Aaron : viewforum.php?f=1

Please let me know!

tercio
Posts: 3
Joined: March 18th, 2020, 4:51 am

Re: RESTful API calls to csweb server

Post by tercio » March 25th, 2020, 4:59 am

josh wrote:
March 18th, 2020, 7:29 am
CSWeb has a REST API. It is documented in csweb/src/api/app/swagger.json. That file is best using the tools here https://swagger.io/

The challenging part of what you want to do it is extracting the data from the CSPro text format that is stored in the database. You need to use the start positions and lengths of elements in the data dictionary to take substrings from the data and convert them to numbers. There are a couple of open source projects that have already done this that you should look at:

https://github.com/IstatCooperation/CSProDashboard
https://medium.com/@nahomt/working-with ... 6161b84ffa
Dear Josh,

Thank you for your promt reply, and sorry for the late reply from my side.
I really appreciate for the details you provided. I successfully found the CSWEB REST API documentation, however I will create the dashboard form the data straight from the csweb database. Many of the fields I need for the dashboard are ID Items, like to see the total number of interviews, number of interviews per province, number of interviews by area of residence (Rural, Urban), etc. As per the substrings, I am thinking of implementing a real time search engine, and for that I have already tested the use of substrings from the 'questionnaire' field in csweb, to return the specific fields I need. I am using php for now, with a database connection file, instead of using API calls, because I struggled with API authentication/authorisation.

My next goal is to find inconsistencies with the syncronized data and provide a report to subject matter specialists.

Kind regards,
Tércio

tercio
Posts: 3
Joined: March 18th, 2020, 4:51 am

Re: RESTful API calls to csweb server

Post by tercio » March 25th, 2020, 5:06 am

htuser wrote:
March 18th, 2020, 1:42 pm
Dear Tercio and Josh,
I intend also to use best in class dashboard https://superset.apache.org/ for live reporting of:
a) Census/Survey data;
b) Geo-enableb monitoring and evaluation system;
c) or whatever information system implemented using CSPro;

But, always waiting automatic data parsing on CSWeb (relational schema support as requested long ago, and also by Tercio);
I already tested CSpro2SQL, but it's limited...
I've just ask Aaron : viewforum.php?f=1

Please let me know!
Hi htsuer,

Thank you for your feedback, which I really appreciate. I saw the link, however the requirement is a dashboard that is created by the local team, and this will be used for all the surveys and censuses using Cspro.

Regards,
Tércio

Post Reply