Posts are my own opinionated views, this blog serves as a journal for both technical and non-technical notes. As well as documenting attempts to replicate and automate intelligent human behaviors.

Non-parametric

Modern methods include k-nearest neighbors(KNN), support vector machines(SVM), gaussian processes, kernel functions, gradient-boosted decision trees and etc.

Future: use cases after year 2020

Parametric

Linear models(MLP), ensemble methods, neural networks and multimodal networks.

Future: operations efficiency(factorized convolution, efficient attention mechanisms), training efficiency, parameters efficiency, inference efficiency

Optimization And Hardware-Acceleration

Pipeline approach: network pruning, low-bit quantization and weights clustering. Porting to distributed GPU clusters.

Future: model architectures that are well suited for compression, take cues from Nature’s design(DNA double helix, nervous system).

Cross-Platform Development

Target platforms include desktop, web, mobile and micro-controllers.

Scaling Models Beyond >1 Trillion and <1 Million Parameters

Datacenter or local on-device.

Future: performance/efficiency tradeoffs

MLOps

Plethora of tools such as Istio, Knative, Argo Project, Tekton Pipelines, Seldon, KubeFlow and many more. Includes pipelines for data processing, model CI/CD/CP/CM/CT/CE and finally deployment to target platforms.

Future: keep the toolbox simple