Blog

Guest blog post by Donald Atha, Software Architect, BetterCloud

One of the biggest challenges facing IT teams today is the increase in tedious and repetitive tasks driven by the explosive growth in SaaS applications over the last 10 years. Companies are using more and more SaaS products, which presents challenges when it comes to managing employee onboarding and offboarding, managing licenses and permissions, and resolving help desk tickets. At BetterCloud, we give IT teams the tools to reduce the amount of manual work on their plate with automations and configurable workflows. As the number of SaaS applications grows, our customers rely on our library of SaaS integrations to also grow.

Testing OpenAPI Generators for Rapid Integration

Recently the Innovation Team within BetterCloud, a team tasked with discovering and testing new capabilities that accelerate innovations, took on the challenge of prototyping a solution that would allow us to more rapidly integrate with various SaaS applications and scale that solution to onboard hundreds of API providers. The team agreed early on in order to reach the ambitions of the business, we would need to automate much of the process. The team also agreed that this would be possible because much of adding a new integration is the same and that the OpenAPI Specification would allow us to automate pieces of this by using existing community generators. 

To fully understand the problem, it helps to understand what it means to add an integration at Bettercloud and a bit about our product. BetterCloud’s workflow engine automates processes like employee onboarding by detecting when a new employee is hired and subsequently granting access to the various SaaS applications required for their role. This onboarding scenario is just one of many use cases we support. All these use cases invariably necessitate ingesting and storing data from the 3rd party provider and updating the provider’s state. For instance, in the onboarding use case, the ingestion process fetches employee data, detects a change in status, and triggers the onboarding workflow. This workflow calls the provider’s APIs to create accounts or grant access to shared resources for the employee—functionality we internally refer to as “actions.”

When embarking on adding a new integration, BetterCloud begins by validating that the provider has the necessary APIs to support our use cases. This crucial step sets the stage for efficiently accessing required data and assessing whether the provider’s APIs are compatible with our action use cases. The OpenAPI Specification provides a consolidated view of this information.

Reducing the Amount of Code Written by Using the OpenAPI Specification

Once the API is understood, the next phase is implementing the ingestion logic. This phase involves calling the provider’s APIs to fetch and store all necessary data within BetterCloud’s infrastructure. The team has explored two methods to coordinate this, both leveraging the OpenAPI Specification. The intent with both approaches is to reduce the amount of code that a developer must write when adding a new integration. The first involves code-generating an HTTP client in Java using the OpenAPI Java code generator, while the other approach uses extensions to markup the spec, enabling the configuration of the ingestion logic. This marked-up spec feeds into a general-purpose engine that implements the logic based on the extensions. This covers the ingestion of data from the APIs, but once acquired, it must be stored to serve our applications. 

BetterCloud utilizes a SQL database to store the entities we ingest. To create the schemas, the team employs the OpenAPI MySQL generator, Flyway, and Terraform. These tools facilitate the creation and configuration of a database for the new provider. The team was able to slightly modify the existing mustache templates of the MySQL generator in order to add our internal fields such as customer id. Since both the ingestion data models and the MySQL data schemas derive from the OpenAPI Specification’s entities, automation of this process becomes straightforward as the data models are aligned. With just an OpenAPI definition, we can generate a new database and its schemas at the click of a button in all of our environments!

Conclusion – Better Scaling the OpenAPI Specification, Better Value

We anticipate that this approach will allow us to bootstrap the ingestion portion of adding a new integration. By automating and code-genning common elements, the team can concentrate on the unique aspects of each new integration. Much like how we at BetterCloud automate workflows for IT teams to allow them to scale the impact of their work, the OpenAPI Specification has given us the tools to build our own automations that scale the value we can provide to our customers.

I can’t believe it’s been ten years since I first started attending GlueCon. This event has always been on my “must-attend conference list” because of its highly curated content, focusing on cutting-edge technology that is poised to transform the web. Throughout the years, I’ve had the opportunity to represent a few different companies at Gluecon leading community teams. However, this year is different. I was unfortunately laid off in March, and now I find myself in search of my next exciting challenge.

A conference like Gluecon is the perfect place for me to network and connect with innovative companies.

Back in 2013, when I first attended Gluecon, the conference was abuzz with discussions about web services and API standards. As a developer evangelist for Context.io, an email data API, I found Gluecon to be the ideal gathering to meet fellow developers and individuals keen on exploring new APIs.

This year, the main topic of conversation was WebAssembly.

For those unfamiliar with WebAssembly, it is a binary instruction format specifically designed for the web. Its purpose is to enable high-performance code execution within web browsers. These low-level languages execute on the web with near-native performance, revolutionizing the way applications are built.

It reminds me a little of when Docker was first released and it was apparent to everyone how useful containers are. In fact, Solomon Hykes, the co-founder of Docker, famously tweeted that if WASM had existed in 2008, Docker would never have been necessary (https://twitter.com/solomonstre/status/1111004913222324225).

The second time Solomon presented Docker publicly was 10 years ago at Gluecon, along with Jeff Lindsey. Jeff referred to Solomon’s quote during his talk at Gluecon titled “Futuristic Architectures: Session Backends and WebAssembly Components.” In his talk, Jeff brilliantly illustrated how standard WASM components will lead to innovative app architectures. It was a delight to catch up with Jeff, as the last time we crossed paths was in Paris at the dotScale conference back in 2014. He was showing dotScale Dokku, which began… as Jeff’s Docker demo for his Gluecon talk the previous year.

Another remarkable session was the fireside chat between Alex Williams, from The New Stack, and Matt Butcher, founder and CEO of Fermyon. Alex has been covering the most intriguing emerging technologies since his days at Programmable Web (RIP), and his conversation with Matt was no exception. Hearing about how the benefits of WASM, like cross-language compatibility and fast execution times, are being implemented by Fermyon in their new cloud offering was very exciting.

Not every talk at Gluecon was about WebAssembly. Any developer would have enjoyed Jason Harmon’s talk focused on internal API best practices. Treating APIs like products vs. a feature of a product leads to good developer experience, faster production cycles, and fewer headaches. Jason is currently the CTO of Stoplight.io, who went through the Techstars Cloud accelerator in 2015. Editor’s note: Jason was also a key figure in the early days of the OpenAPI Initiative!

It was fun bumping into Samy Fodil, who I met when his startup Sage Hero was in Techstars Cloud in 2016. Samy was at Gluecon this year talking about his current startup Taubyte and, yep, WebAssembly!

Gluecon always closes with some amazing keynotes. Unfortunately, I had to miss this year to get set up for Mile High Startups & Music, the monthly tech event I promote featuring Denver’s best local musicians. (Big shout out to Elastic for supporting both Gluecon and Mile High Startups & Music!)

I hope to see you there next year!

The Open Geospatial Consortium (OGC) recently announced that OGC API – Tiles was adopted as an official OGC Standard. The OGC is a collective problem-solving community of experts from more than 500 businesses, government agencies, research organizations, and universities representing hundreds of thousands of geospatial professionals driven to make geospatial (location) information and services FAIR – Findable, Accessible, Interoperable, and Reusable.

The OGC API - Tiles Standard defines building blocks for creating Web APIs that support the retrieval of geospatial information as tiles. Different forms of geospatial information are supported, such as tiles of vector features (“vector tiles”), maps, imagery, and other types of geospatial information.

The new standard focuses on simple reusable REST API building blocks that can be described using the OpenAPI Specification.

Who uses the Open Geospatial Consortium and why? Can you give an example?

Standards by the Open Geospatial Consortium are used by hundreds of organizations to publish millions of datasets, as reported by GeoSeer.net. Many of the organizations are OGC Members that all actively participate in a consensus process that designs and publishes standard specifications which improve interoperability. A few of the domains that make use of OGC Standards are Aviation, Built Environment & 3D, Business Intelligence, Government & Spatial Data Infrastructure, Energy & Utilities, and many more. 

A great example of those standards being used would be in the aviation industry like EUROCONTROL, the European Air control agency and the Federal Aviation Administration (FAA) in the United States. Those teams publish a lot of their data using OGC Standards, specifically the Geography Markup Language (GML) which they have implemented in the Aeronautical Information Exchange Model (AIXM). Another example of the use of the standards is the OGC API – Features Standard which has been adopted as a Good Practice for implementing download services that are compliant with the European Union’s INSPIRE Directive, which positively affects hundreds of millions of EU citizens across the continent.

What’s new about the release of OGC API – Tiles? How will this be used? Who should use this?

We are really excited about it. The OGC API – Tiles Standard has been in the works for a number of years and now it is here. The OGC API defines building blocks for creating Web APIs that support the retrieval of geospatial information in tiles, basically little image chips that can be streamed with nearby chips to show a map. There are many great things about it, some special features include map tiles and vector tiles. Rather than the end user having to retrieve a large data set for the whole world, they can just retrieve that single tile. They can then use the identifier for that tile to collaborate with colleagues. Those in environmental sciences might retrieve a tile from other specialties and vice-versa. Now, instead of transferring terabytes of data across a network, it is just a subset. 

We have standardized models for Tile Matrices, we have a registry where various organizations can agree on a set of tiling schemes to use together. There have been significant efficiency and cost savings seen across the board. Transmitting complete datasets over a network can incur some charges, so transmitting only relevant subsets offers cost savings. Also, performance has improved. For tiles that do not change, it is now possible to provide a cached tile. We recently hosted a code sprint in Brussels where developers came together from across the globe, and within a short time they were able to implement this standard, and simply just used the implementations through various client applications. 

I’ll point out, all of that was made possible because the OGC API – Tiles Standard uses the OpenAPI Specification, which makes it attainable for web components to describe the capabilities for the resources, schemas, and so much more. In the previous generation of web service standards, it required developers to interpret the requirements of a standard. Now it is possible for some of that role to be done by the applications themselves. Apps can interpret what the API is intended to do. This takes away the burden from the developers and it leads to very happy developers!

What is the advantage of open source with spatial information?

Open standards by the OGC are used by both open source and proprietary software products; they do that to make Geospatial information more findable, accessible, interoperable, and reusable. One key thing about open standards and open source software is that they reduce the risk of vendor lock-in. With vendor lock-in, an organization gets tied to a single vendor and they must obtain products from that vendor. With open standards and open source software, it certainly reduces the risk of this and organizations do not have to buy from a single vendor. Furthermore, some common parts of commercial software come from open source to reduce the cost of developing some of that fundamental technology that can be shared; for example, the open source GDAL library is used by several commercial software products.

Why do you support the OpenAPI Specification? What are the main advantages?

The OpenAPI Specification makes it possible for developers to automatically create source code through code generators, by simply parsing an API definition document that complies with the OpenAPI Specification. In the past, most creation of source code required developers to manually read and interpret API definition documents, which led to human error and was certain to take longer to implement interfaces. Now, a lot of the interpretation is done by the application and then the human developer joins to codify the business logic. Moreover, using OpenAPI means that all of our APIs are defined in a consistent way so that the use of the APIs is predictable for all developers.

The efficiency gain for an organization is incredible. It is one of the key reasons why the OpenAPI Specification has been looked upon favorably by the OGC community.

OGC develops open standards, so everyone has the opportunity to provide some input to the standards. Other organizations do not have to wonder if there are some restrictions to access to the standards. But also, organizations can feed requirements in, as a community both the OGC and the Linux Foundation, which houses the OpenAPI Initiative and looks after the OpenAPI Specification, have done an excellent job of involving everyone across industry. There is a lot of feedback that is included or at least considered in the design of those standards. Everyone in the community has a chance to include input into those standards.

What is the best way to get involved in the OGC?

The OGC runs three member meetings a year, in different parts of the world, and with hundreds of participants. Now that travel restrictions are easing up, those meetings are now hybrid meetings with remote participants being supported. 

OGC working groups use those meetings to gather needs and specs. In between those meetings, there are a series of working group get-togethers. For anyone that is interested in participating, I would recommend going to the OGC website and have a look at the information about membership, there is also a list of working groups. We have domain working groups in defense, aviation, meteorology, and more. We also have other working groups that focus on specification, like OGC API – Tiles and OGC API – Features.  

The next OGC meeting will be from 20th to 24th February 2023, hosted by the European Space Agency in Frascati, Italy. That will be the first of three taking place in 2023, there will be another in June in Huntsville, Alabama, and another in September in Singapore.

I would also like to mention, the OGC runs several initiatives, we run innovation initiatives, where members can join in. These are run by the Collaborative Solutions and Innovations team in OGC, those initiatives are funded by the OGC membership, and they provide an opportunity for research and development. I encourage anyone interested to look at the OGC website and participate in these activities. 

---

OpenAPI Resources

To learn more about participating in the evolution of the OpenAPI Specification: https://www.openapis.org/participate/how-to-contribute

•  Become a Member
• OpenAPI Specification Twitter
• OpenAPI Specification GitHub – Get started immediately!
• Share your OpenAPI Spec v3 Implementations

About the OpenAPI Initiative

The OpenAPI Initiative (OAI) was created by a consortium of forward-looking industry experts who recognize the immense value of standardizing on how APIs are described. As an open governance structure under the Linux Foundation, the OAI is focused on creating, evolving and promoting a vendor neutral description format. The OpenAPI Specification was originally based on the Swagger Specification, donated by SmartBear Software. To get involved with the OpenAPI Initiative, please visit https://www.openapis.org

About Linux Foundation

Founded in 2000, the Linux Foundation is supported by more than 1,000 members and is the world’s leading home for collaboration on open source software, open standards, open data, and open hardware. Linux Foundation projects like Linux, Kubernetes, Node.js and more are considered critical to the development of the world’s most important infrastructure. Its development methodology leverages established best practices and addresses the needs of contributors, users and solution providers to create sustainable models for open collaboration. For more information, please visit us at linuxfoundation.org.

The OpenAPI Initiative, the consortium of forward-looking industry experts focused on evolving and implementing the OpenAPI Specification (OAS), is announcing that Bump.sh has joined as a new member. Welcome!

Bump.sh is an API contract management platform that helps document and tracks APIs: identifying what changes in APIs structure, and keeping developers up to date across an organization. Bump.sh acts as a single source of truth with information that remains up to date and changelogs.

The company has raised $4 million in funding led by Galion.exe and Bpifrance’s Digital Venture fund. 

“We believe schemas are the cornerstone of future-proof API development. As a vendor, it is our duty to contribute to and keep up to date with the latest specifications,” said Sébastien Charrier, CEO of Bump.sh. “Joining the OpenAPI community will help us better support our customers and contribute to the direction of OpenAPI Specification development.”

To learn more about Bump.sh, please visit: https://bump.sh/ 

Want to become a member of the OpenAPI Initiative? Find more information here: https://www.openapis.org/membership/join 

OpenAPI Resources

To learn more about participating in the evolution of the OpenAPI Specification: https://www.openapis.org/participate/how-to-contribute

●   Become a Member

●   OpenAPI Specification Twitter

●   OpenAPI Specification GitHub – Get started immediately!

●   Share your OpenAPI Spec v3 Implementations

About the OpenAPI Initiative

The OpenAPI Initiative (OAI) was created by a consortium of forward-looking industry experts who recognize the immense value of standardizing on how APIs are described. As an open governance structure under the Linux Foundation, the OAI is focused on creating, evolving and promoting a vendor neutral description format. The OpenAPI Specification was originally based on the Swagger Specification, donated by SmartBear Software. To get involved with the OpenAPI Initiative, please visit https://www.openapis.org

About Linux Foundation

Founded in 2000, the Linux Foundation is supported by more than 1,000 members and is the world’s leading home for collaboration on open source software, open standards, open data, and open hardware. Linux Foundation projects like Linux, Kubernetes, Node.js and more are considered critical to the development of the world’s most important infrastructure. Its development methodology leverages established best practices and addresses the needs of contributors, users and solution providers to create sustainable models for open collaboration. For more information, please visit us at linuxfoundation.org.