Classification on Letters From The Wild Side

Flutter Web, TensorFlow and PyTorch Project

Sun, 27 Jun 2021 00:00:00 +0000

Last modified: May-29-2022, 02:40PM +08

Progressive Web App

Progressive web app(PWA), a term that is increasingly common whenever one does a search on the web in 2021. To date, there have been over 200, 000 applications in the Play store built using Flutter. At it’s core, a PWA is an application software that runs on a web server and accessed through a client such as a browser.

The PWA alternative paradigm to traditional app development is made possible by the ongoing work on large number of modern web APIs such as Cache, WebGL, WebAssembly, Bluetooth, File System, IndexedDB, Service Workers and many more.

Over the last few decades, cybersecurity is being increasingly recognized as critical infrastructure, the Covid-19 pandemic further accelerated the shift towards the digital landscape in an attempt to return to normalcy. The 2021 Global Threat Report by Crowdstrike provides details on how adversaries exploit weaknesses present in current day business and government infrastructures. Security is now a first-class citizen in PWAs as communications and/or data must be served over TLS connections.

A PWA looks, feels and navigate like any other web page with nested URLs or it can function as a single page application. It is also installable and executes using the browser runtime. Additional functionalities include having the ability to work offline and access to device hardware, e.g., camera, microphone or GPU(s), that are traditionally available only to native applications. The app developer is also able to embed PWA within a web page or vice versa using WebView on mobile, resulting in a hybrid framework.

The PWA is not limited to mobile platform, even though there is increasing adoption on smart watches. It is also compatible across platforms, look beautiful and responsive at different resolutions and/or orientations. It does this by treating different resolutions and platforms as multiple independent states, resulting in a single codebase, which greatly streamlined rapid iteration of feature development.

Being browser and platform independent, all you need is a stable internet connection which enable high interactivity, performance(FPS) and low latency responses for simple through medium complexity use cases. A PWA can be configured to work with any input type including touch, mouse, keyboard, audio or gestures.

The PWA paradigm trades native performance for flexibility.

Flutter For The Web

A PWA framework in the spotlight for web development is Flutter Web which hit the stable milestone in 2021.

However, it takes high cognitive effort in navigating complex low-level APIs for intermediate and advanced usage. As with any new and exciting framework, Flutter attracts droves of developers in implementing their own Flutter port of exiting applications, only to be met with lacking documentation and/or over-simplified examples that do not translate to real world use cases.

Usability issues aren’t unique to Flutter Web. Coming over from TensorFlow 1.x, intermediate and advanced usage also experienced the same kind of brick wall once the developer crossed the novice speed limit. With TensorFlow 2.x, the engineering team adopted Keras as their high level API with an emphasis on progressive disclosure of complexity and greatly improved usability.

In my neutral opinion, TensorFlow 2.x and Snapcraft serve as good starting points for communicating user/reference guides, targeting different expertise levels. As such, I have newfound appreciation for well communicated technical documentations. As someone starting with zero experience in web development and web technologies, Flutter Web represents an enormous challenge with a tremendous investment in cognitive effort. Previously, Streamlit was my go-to for rapid experimentation. Jumping from Streamlit to full-fledged Flutter Web is akin to bungee jumping in Grand Canyon, a straight plunge to rock bottom.

Not recommended for new, aspiring web developers with restricted time allowance as APIs implementation such as Navigator 2.0 can get low level and filled with boilerplate for intermediate and advanced use cases. There is significant effort in reviewing third party alternatives where several packages are replicating similar use cases for complex APIs. Due to the complexities in modern network of web technologies and native platform APIs, community contributions are in great need.

It is also due to this patchwork of volunteers and industry that allow a PWA built with Flutter Web to exhibit near native performance across different platforms and modern W3C-compliant browsers. A Flutter Web PWA just works, no need for app stores, no hard requirement to download or install any executable.

The real testament to Flutter framework is emulating Wechat which serve over 1 billion users and represents a super app, housing smaller apps within its ecosystem.

Modern Browser As A General Purpose Computing Platform

Evolution from fetching web pages, reading emails to crunching computation in a secure and sandboxed environment. Modern browsers greatly enhanced productivity and entertainment with plugins ecosystem and a growing body of Web APIs. A general-purpose modern browser represents an international community effort in a pursuit of a fair, open and privacy-preserving high accessibility tool. Recent functionalities include programming sandbox(IDEs), screen casting and machine learning powered tools such as autocomplete search.

Symbiosis Of Flutter And TensorFlow

As a tinkerer, there’s an itch to satisfy after witnessing the exponential advancement in modern technologies. Hence, the inspiration to create a tool that’s designed to initiate creative and/or problem solving processes, reducing the user’s cognitive inertia with productive work. The fruition of this project is evident by well defined interfaces of public facing APIs in seemingly unrelated fields(Flutter Web, TensorFlow, TensorFlowJS, TFX) across different languages(Python, C/C++/CUDA, JavaScript/TypeScript, Dart).

Initial machine learning(ML) models were written with TFX pipelines, the SavedModel outputs were further converted to JSON format to be compatible with the browser. Separate JavaScript/TypeScript scripts would contain the logic in loading the converted browser-compatible models and handling inference requests.

These inference scripts are executed as callbacks from Dart classes upon accepting user inputs in the app UI. For low complexity models that have just a single parameter and accept a single input, response time are <1 second. For medium complexity models that have multiple parameters and accept a single input, response time are also <1 second. For high complexity models that require loading from content delivery networks(CDNs) and accept open-ended inputs such as text, audio, image or video, response time range from 3 to 10 seconds.

Preliminary exploration suggest that these technologies are fully compatible and worth further investments into advanced capabilities of the humble web browser.

What’s Next?

Future roadmap include expansion of low, medium and high complexity models with strict performance restrictions. Extending application to process text, audio, image and video data efficiently. Explore different problem domains in related fields of computer vision, natural language processing and machine-generated content.

Summary

General purpose ML toolbox that is cross-platform and readily accessible through the internet.

PWAs will continue to proliferate due to it’s flexibility
Flutter Web hits stable milestone for production use
Modern W3C-compliant browser with exciting APIs
Flutter + machine learning frameworks = UI meets AI
Roadmap

ML Engineering

Tue, 08 Jun 2021 00:00:00 +0000

Last modified: Dec-14-2021, 10:35PM +08

From Manual To Semi-Automatic

Before the advent of the concept “MLOps”, getting a single machine learning(ML) model to production was tedious and belaboring. Every single detail pertaining to the inputs, model server, training and inference have to be defined explicitly. This is to ensure the input tensors follow a strict requirement for them to be processed by user defined functions.

To serve a single model, these predefined configurations have to be under version control as the ML field and software ecosystem is accelerating at near exponential speeds. In addition to the model, version control has to be applied to the training data as well as the software infrastructure that is used to host the model. A working production pipeline is like a moving train loading and offloading compartments to keep up with cutting-edge development.

After the release of ImageNet dataset, there was tremendous effort poured into surpassing the human baseline. In the early 2010s, that baseline was exceeded with the combination of readily available data, open-source frameworks and modern computing resources that can be bought off the shelf. However, being proficient in these resources was restricted to experts and those within the technical community. Developer sanity was largely dependant on up-to-date documentation or comments within the source where documentation was absent.

In the mid 2010s, a number of Deep Learning(DL) frameworks were designed to unify the common primitives in building these DL models. These include TensorFlow, Keras, Pytorch, Apache MXNet and many others.

To tackle the problem of productionizing models, one of the solutions explored was the usage of Docker containers, to package both the dependencies and the actual model as lightweight components that can be easily shared through a public repository. This approach greatly democratize the deployment of DL models to common hosting providers like the public clouds or in-house servers.

The natural progression in using Docker containers meant the inclusion of shell scripts, cron jobs and triggers that allow the automation of the entire ML pipeline. Docker-based workflows gave developers access to version controlled resources locally on their laptops and globally across different time zones.

Components-Based Workflow

For organizations that need to scale to millions of containers in production, the de facto solution include container orchestration platforms such as Kubernetes. The platform allows hundreds and thousands of engineers to collaborate on different levels of a complex ML system. This ranges from low level implementation of hardware drivers to the high level design of user-interfaces such as click-and-drag block diagrams.

The low-code or no-code approach is an industry effort to lower the cognitive strain in designing complex ML models. The design and implementation of mission-critical models requires non-trivial engineering efforts, so why should their deployment be unnecessarily complex?

Behind the scenes of the components-based workflow lies Kubernetes applications such as Argo Workflows, Tekton as well as many others. These applications specify steps in a ML pipeline as containers that can spun up sequentially or in parallel. These steps can be expressed as a directed acyclic graph(DAG), which can be version controlled and compiled for export to different hardware architectures.

Initially, we had manual design, hand-tuned and hand-crafted models without A/B testing because deployment of new models simply could not keep up with the development of a core application(4~6 weeks cycle). Now we can churn dozens of models daily in parallel, set to trigger on arrival of new data or based on adjacent/over-lapping time windows. The models that passed evaluation are then uploaded to a model repository for further downstream processes.

Cautionary Tales

A majority of kubernetes applications are rather new to the scene, many more are emerging to solve critical issues pertaining to storage, security, networking and other peripherals. Choosing the right software stack requires an in-depth technical review of existing solutions with respect to dimensions of correctness, latency and costs.

At the SME scale, one single competent ML engineer is the bare requirement for a sufficiently complex ML system, serving requests up to the number of CPU cores procured with default settings.

At the enterprise scale, ML engineering is not well suited to be an one-man job, but rather spread across different teams with each being a subject matter expert on their domains.

What’s Next

Currently working on a regression pipeline, targeting TensorFlow.js models to be deployed in a Flutter application, hosted by Firebase. The pipeline is designed to be agnostic to regression problem domains. Future regression tasks include cryptocurrency market size, health monitoring, renewable energy forecasts and EV tank-to-wheel efficiency(70~90%).

Other pipelines include tasks under the pillars of ML:

classification
density-estimation
dimensionality-reduction

Pipelines for generative models are in the roadmap as well.

Incorporating accelerators such as GPUs or TPUs into pipeline to further parallelize existing workflows.