Data Export

Merging files of different data levels

Survey Solutions produces export data files separately for each data level. This can be for example households, persons, plots, crops, etc. The process is completely automatic and not configurable by the user. If you need to combine information from different data levels (for example bring household characteristics to the personal level) you can utilize a statistical package to post-process your data. This is a basic operation supported by most packages, such as SAS, SPSS, Stata, R and others, though the exact terms may differ by package.

January 7, 2019

Paradata file format

Every survey collected in Survey Solutions is supplemented with a paradata file. No actions are needed from the questionnaire designers or headquarters team to collect it, it is produced automatically by the Survey Solutions software. The paradata files describe the process of data collection. They explain how the data was entered, detailing all edits, who and when undertaken them. These files may be large and most conveniently processed using specialized statistical packages, rather than general purpose tools.

December 28, 2018

System generated files

Every archive of data exported from the Survey Solutions Headquarters contains among the user data the following system-generated data files: interview__actions; interview__comments; interview__diagnostics; interview__errors. The data files are also accompanied by an automatically generated file named export__readme.txt. This file documents which version of the Survey Solutions software was used to export the data, when the export was done, and placement of variables into the data files. Here is an example of such a file that may be accompanying, e.

December 12, 2018

Interview diagnostics file

interview__diagnostics file is a special service file, automatically produced with every data export, that contains useful information for assessing the quality of interviews: number of invalid entities (invalid answers and errors in static texts), number of questions with comments, duration of the interview etc. This is especially useful for user-written monitoring systems that work alongside Survey Solutions and continually monitor the quality of incoming data by periodically exporting the data and building customized reports.

October 19, 2018

Export of calculated variables

Calculated variables are commonly used in CAPI questionnaires to implement complex checks or indicate to the interviewer some derived information. For example, one may create a variable that calculates the number of children in the household, numChildren = Persons.Count(person => person.age < 18) and use this variable in subsequent questions to check if the number of children in a household exceeds a specific threshold. Starting with version 5.24 of Survey Solutions these variables will be automatically included in the export data.

September 28, 2018

Export to cloud storage

Binary data files, such as pictures and audio recordings are often large and can be time-consuming to download. One can now opt for pushing the data to own cloud storage, rather than downloading directly. The following cloud storage repositories are supported: DropBox; Google Drive; OneDrive. The direct download of binary data in the form of a single large zip-archive remains available. To push the data to the cloud storage, select the corresponding option in the download dialog, then authenticate yourself with the corresponding cloud storage provider:

April 2, 2018

Interview statuses (actions)

Survey Solutions tracks the actions performed on interviews (such as change of status or other major actions) using the following pre-defined actions: SupervisorAssigned InterviewerAssigned FirstAnswerSet Completed Restarted ApprovedBySupervisor ApprovedByHeadquarter RejectedBySupervisor RejectedByHeadquarter Restored UnapprovedByHeadquarter Created Resumed Paused TranslationSwitched OpenedBySupervisor ClosedBySupervisor You can encounter mentioning of these actions in the interview__actions file and in paradata along with the timestamp when the action was taken and accounts involved.

February 7, 2018

Missing values

Survey Solutions recognizes two situations of missingness: the value is missing because the logic of the questionnaire is disabling (skipping) this question. There is nothing that the interviewer can do to enter a value in such a field and in some cases even to see the question and its options. the value is missing despite the question is enabled (not skipped). This could be because of any of the following: interviewer hasn’t reached this point in the interview; interviewer missed the question, didn’t ask it and hence no answer was recorded; none of the options was suitable to accommodate the respondent’s answer and hence the interviewer didn’t record the answer; other similar situations.

June 13, 2017

Stata export file format

When Survey Solutions v5.18 is writing an exported data file in Stata format it is producing it according to Stata 14 data file specification, available from StataCorp LLC. Stata versions prior to version 14.0 (e.g. 12.0, 13.1, etc) cannot open these files because these older versions of the software are not aware of the newer file format. The message that the user will see on the screen varies, and may look, for example like the following:

April 11, 2017

Rosters-Export File Anatomy

Recall that each exported file corresponds with a level of observation in the questionnaire. Rosters and their source question are exported as follows: Source question: Export file at the questionnaire level (corresponds with the name of the questionnaire) Roster: Export file at the roster level (corresponds with the roster name) List Roster In the export file at the questionnaire level, a list roster source question generates as many columns as the defined maximum number of list elements.

November 9, 2016