We have reached the “360° Partner” with Bonitasoft
July 28, 2021
Using Bonita BPM to automate and manage the generation, publication and delivery of high-resolution maps and three-dimensional models using images captured by Unmanned Aerial Vehicles (“Drones”)”
Hello everyone. My name is Quauhtli Martinez. During the last decade, I’ve been working on Enterprise Application Integration (mainly Service Oriented Architecture design and implementation) and Business Process Management (BPM). Right now I work for SOA Software Factory (www.soaswfactory.com) a Mexican company specialized in BPM projects and Bonitasoft partner since 2012.
In addition of working on BPM projects, on recent years I have been playing and working with Unmanned Aerial Vehicles (aka “Drones”). With two friends, I founded the Mexican startup Droneware (www.droneware.mx) a company specialized in design and development of drones for use in various commercial applications such as aerial photography, high-definition mapping and three-dimensional model generation, crop surveying, solar panels inspection and power lines and towers for telecommunications as well, among others.
I consider both issues (BPM and Drones) very interesting and hot topics right now, so I’ve been trying to develop an application in which I can mix both of them. Derived from the experience I have had in using drones to capture geo-referenced aerial images, process them and sending the final results to our clients, arises the idea of making an application based on a business process, in which we can assistance people who already have their own mapping drone setup but don’t know how to process the captured geo-referenced images and to generate important information like maps, high-resolution three-dimensional models which is very helpful in some applications like architectural, mining exploration, building construction and so on.
So this idea stems from the assumption that a person who has such drone setup do not know how to generate 3D Models or maps, it’s not in their interest, don’t want to purchase software licenses and / or is unwilling to invest time to process the captured images and generate results and delivering them to its own customers, but do knows how to flight and to take some geo-referenced shoots using drones (which most of the cases is this way).
Generating high-definition maps and three-dimensional models is based on a business process, in short, as follows: 1) a physical inspection is performed on site; 2) flight missions and parameters are designed for taking geo-referenced imagery; 3) autonomous flight and capture geo-referenced images is done; 4) images are processed; and finally, 5) images are generated and processed to generate digital map image (usually GeoTIFF images) and 3D Model objects (OBJ wireframe+texture objects).
To perform each of the steps in the procedure described above, different applications are used; each one requires some knowledge and a more or less complex expertise. In this article, I wrote for the Mexican magazine “Software Guru” you can read more about the detailed procedure and the software used at each step, only available in Spanish).
On the other hand, applications that are used to process and generate the final images usually require the payment of an annual license round in thousands of dollars annually, plus time and high capacity computers due of the power you need to process them. The cost of these licenses can pay for itself, if the software is used very often, however, for casual processing, the cost will never be recovered.
The development of this application is in progress, not even Beta version yet; however, this article seeks to show how the awesome Version 7 of Bonita BPM has been used to solve this problem. New features Bonita BPM allow exposing the process as end-user oriented business process application; in this case, this is essential because the final application will be aimed at users where simplicity and usability will be crucial.
The first step in its development is the business process modeling through the BPMN 2.0 notation standard (using Bonita BPM Editor). The process, named “UAV Data Processing 1.0”, was modeled based on the steps mentioned in the preceding paragraphs. As you can see in the following image, it all begins with 2 system activities (ie, “non-human”), in which the value associated with the process creator is stored in a business variable and also the value of the current activity. Both values are then used to determine the status of applications through a Custom Page described in the next paragraphs.[/az_column_text][az_lightbox_image image=”452″][az_column_text]
During the progress of the process instance, the application is sending notifications to the participants so they always know the status of every request without having to enter the application. This is done through the Mail Connector at the beginning of every human activity as you can see in the following images.
Prior to these human activities, the process has an associated instantiation Form in which applicant (“Requester” Group and Role) captures the flight information and upload images to the application in a zip file stored automatically in the document repository from Bonita BPM engine as a “Document” which is sent for processing when the user submits the form.
As you can see in the previous images, a Bonita BPM Custom Application was created to give a more “friendly” look-and-feel to users that submit the requests. The form (“Page”) associated with the instantiation activity is settled in this application. This Page was originally built from Bonita Studio to take advantage of the UI Editor, but once developed, was exported and modified to perform the instantiation process through the submit button without having to be within the Bonita BPM Portal. Processes Instantiation Buttons that are created in forms developed from Bonita Studio use part of the Bonita portal URL to extract the ID and name of the process and then made the call to the instantiation function through the Bonita BPM REST API so that’s I had to modify it to manually enter the process ID and not acquiring from the Bonita BPM Portal URL. View the selected text in the following image.
Once stored these values, “Data Validation” human activity starts. In this activity, users associated with the group and role “Data Validator” verify the information submitted by applicants in the previous human activity. Verification is performed by downloading the “zip” of images sent by applicants and by the data entered on the form. At this point, the process has two possible paths: the first, where images and information received is correct and suitable for processing; and second, where the images and / or information received are rejected. When the information is incorrect, the process returns to the activity “Service Request Correction”, where the “Requester” user correct the information and sends it back to the validation activity. If the information is correct, the process continues to activity “Data Processing” and role associated with the group “Data Processor” where the user only checks the images before processing.
After images are correctly validated, the generation and publication of results begins (“Results Generation and Publication” Human Activity). The processing of images is performed through the “Agisoft PhotoScan” package outside Bonita BPM. Once done processing, the zip file containing the resulting imagery is uploaded using the form. On the other hand, two URLs are typed by the Data Processor user so that the requester can “pre-view” both the 3D Model and the final high resolution map within the same Custom Page built in Bonita BPM and exposed through the Custom Application with a Popup Window with an embedded iframe without the need of rendering or processing the files by itself.
Those URLs correspond to sites where you can view the three-dimensional model of the processed images. In the first case (the 3D model), OBJ and textured files produced by the Data Processor user through Agisoft PhotoScan outside Bonita BPM are uploaded first to “Sketchfab”. This is a site that has the capabilities of “rendering” OBJ files with textures and convert them into 3D models that can be viewed via the Web browsers. On the other hand, GeoTIFF images (corresponding to the hig-res map) are uploaded to a local geo-map server powered by “GeoServer”, an open source server for sharing geospatial data via Web.
Actually this is the step that adds value to this idea. Users (requesters) do not have to know how to process images to produce results so they concentrate only in taking good geo-referenced images and we can take care of the rest, using the right software for any kind of results wanted (i.e. 3D Models and high-res maps). See following images.
Once published and uploaded the results, the business process continues to the “Results Download and Validation” human activity where requesters can, on the one hand, download the files generated in the previous activity which usually contains GeoTIFF and OBJ+Texture images for their own processing if desired, or they can also have a preview of the same through the display of the three-dimensional model and the high-resolution map within the same application without processing them.
Downloading results can be done in two ways: 1) through the form associated to the human activity and using the Bonita BPM Portal, and 2) using a custom “Page” (so-called “My Requests”) and installed within the Custom Application. In this page, the user can list their requests and know its current state; when the results are ready, the page automatically displays buttons so the user can download the results and have both three-dimensional model and high-resolution map previews. The interesting part of using these pre-visualizations is that the user can interact with the models, for example can perform rotations, approaches, and measures, without the need of processing them by itself.
Inside the Results Download and Validation Page, the requester may choose to accept the results or reapply for processing. If choose re-process, the user provides feedback within the form and the process instance returns to the previous activity (“Results Generation and Publication”) once submitted. If the user accepts the results, the process instance ends.
Challenges and improvements to have a “publishable” version:
- Develop a marketing site for publishing the application with registration for new users and so on. Very important!
- Allow user to plan routes and generate missions for each flight (maybe using another Business Process where they may receive feedback on the mission planning). BPM-IoT-oriented, totally possible.
- Display real-time flight information (via the API of each drone, such as “DroneKit” from “3D Robotics”).
- Improve UX forms for each activity:
- Add comments (chat type)
- Add document versions management
- Disallow files over a certain size and limit extensions
- Add more “Pages” to the Custom Application so that the end user (“Requester”) does not need to use the full Bonita BPM Portal (called “User Experience” on previous versions of Bonita).
- Automate Results Generation and Publishing activity without human intervention.
- Replace Agisoft PhotoScan and Sketchfab for another OS platform or develop our own
- Automatically obtain the ID and the name of the business process to avoid “hard-coding” it within the process instantiation custom Page
- Improve the content of the notifications so useful information be sent on them such as the name of the flight, status, link system access and so on.
- A lot of UX improvement!
Below is a summary of some features used in this scenario of the new awesome Bonita BPM version 7
Custom Widget development for displaying three-dimensional models and high-resolution map in a “Popup Window
Using a Page Editor to list user Requests, Download Original Photos, know request status and show both 3D Model and Orthomap. This page is published inside a Custom Application for end-users (Requesters)
Forms using REST services, Javascript and local variables
Setting Business variables initial values via the Expression Editor
Setting the Contract values through Activity Operations
Setting output values in the form so the Operations defined in Activities may set Contract values through its Operations
Using Bonita REST API for reading Business Data
Business Data Model definition
Using Business variables in the process based on the Business Data Model
Using Contracts on every Activity so I can store (and validate) the values captured on every Form in the Business Variables
Defining a custom Organization to represent my actual business scenario which includes the Groups, Roles and Users I need on this
Bonita BPM forms integration with external REST Services developed in Java (Jersey Framework)
