More than 1000 form fields and growing: How Waylit uses JSON Schema

Introduction

If you’ve ever filled out a visa application, you know how tedious it is. It’s the same set of questions over, and over, and over again. At WayLit, we help our customers, and their employees make immigration a bit easier. We can’t do much about the government regulations. But we can certainly make the process a little easier.

The standard practice in immigration is to send the government form or an equivalent Word document that collects all the information needed. This is not the best UX one could imagine. We wanted to do better by minimizing the amount of information we asked at a time, and also make the form more adaptive to the beneficiary’s exact immigration situation.

The Development Process

Our first attempt was to use no-code form software along the lines of Google Forms. Unfortunately, these kinds of software do not allow for versioning and evolution of the form as our needs change.

Our second attempt was to use the forms feature from our backend technology — Django. Unfortunately, for long complex forms, embedding the form logic with a good UX proved to be too much of challenge.

Next, we researched some frontend form libraries, such a Formkit. Unfortunately, the schema format is a custom format that has no support outside of Formkit. To validate the data on the backend we’d have to generate a separate schema that needed to be kept in sync. When the two would get out-of-sync, which is inevitable, it could lead to data integrity issues.

We settled on JSON Schema to specify the schema of the data and somehow generate the forms from that schema. We evaluated a number of solutions, and eventually settled on using json-editor. json-editor takes a JSON Schema definition with some extra non-standard properties and generates nice looking adaptive forms.

When evaluating existing JSON Schema based form generators, we had a few criteria in mind: a) it should be mature, b) should be customizable, c) should be actively maintained, and d) as few dependencies as possible. json-editor fit the bill. But there are some other great technologies available for different needs. See them at https://json-schema.org/implementations#web-ui-generation

Understanding the Core Technologies

JSON Schema allows us to express the schema for the data we’re collecting in a portable manner. We can use this schema both on the frontend (browser) and on the backend (server).

json-editor takes this a step further by creating intuitive user interfaces based on JSON Schema definitions. This integration ensures that our questionnaires are not only effective in data collection but also have a nice UI.

The jsonschema Python package plays a crucial role in validating JSON data against the defined JSON Schema. This validation is crucial in maintaining the integrity and reliability of the data collected.

Features and Benefits

We define a single schema for collecting the required information from our user. The authors and maintainers of json-editor library have integrated several third-party libraries such as choices.js, flatpickr, etc. to provide user-friendly features in a declarative manner. For example, the following JSON Schema definition generates the form in the screenshot.

json-editor has several hooks that allow us to customize the form’s looks, behavior, and validation for our needs.

On the backend we use Python jsonschema library with some custom validators to meet our needs.

Limitations

There are some quirks to how json-editor integrates with some third-party libraries. As a result, we have had to do some custom integrations to meet our needs.

json-editor only supports JSON Schema Draft 4 with a few features for Draft 6 and Draft 7. This means we’re missing out on some of the features from the newer drafts.

Currently, changing a form, or generating a new one does need time from the development team. This limits the iteration speed on our questionnaires. However, considering the limitations in the no-code form platforms, it’s a worthwhile tradeoff.

What we do at WayLit

WayLit was founded to provide a better immigration experience for the employees and the HR managers who handle immigration.

Business immigration is a long and complex process in the US. Employers and employees must maintain compliance with regulations. The stakes are pretty high, as non-compliance or missing a filing timeline can cause business disruption.

The existing solutions on the market are geared toward helping attorneys do their work. But we felt the need was equally great on the employer and business side. WayLit brings a more pro-active approach to plan and execute business immigration operations. This ensures operational efficiency, business continuity, and better employee experience.