Author: Fritz

Our experience of the world is multimodal — we see objects, hear sounds, feel the texture, smell odors and taste flavors and then come up to a decision. Multimodal learning suggests that when a number of our senses — visual, auditory, kinesthetic — are being engaged in the processing of information, we understand and remember more. By combining these modes, learners can combine information from different sources.

Continue reading Introduction to Multimodal Deep Learning

In machine learning, a hyperparameter is a configuration variable that’s external to the model and whose value is not estimated from the data given.
Hyperparameters are an essential part of the process of estimating model parameters and are often defined by the practitioner.

When a machine learning algorithm is used for a specific problem, such as using a grid search or a random search algorithm, then you’re actually tuning the hyperparameters of the model to discover the values that result in the most accurate predictions.

Continue reading Introduction to Learning Rates in Machine Learning

Hand detection is a fundamental sub-task of object detection that has historically been hard to implement. But there have been many technological advancements in the field of machine learning and AI in recent years.

One of the core technologies widely used is TensorFlow, an end-to-end open-source platform for machine learning. It has a wide-ranging, flexible ecosystem of tools, libraries, and community resources. With the power of TensorFlow technology, researchers and developers can easily develop and deploy ML-powered applications.

Continue reading Introduction to hand detection in the browser with Handtrack.js and TensorFlow

In this tutorial, we’ll cover how to convert a model in PyTorch to the ONNX format and then load it to Caffe2. We’ll then use Caffe2’s mobile exporter to execute it on mobile.

Continue reading Using ONNX to Transfer Machine Learning Models from PyTorch to Caffe2 and Mobile

Apple’s SwiftUI announcement during last year’s WWDC (2019) was a welcomed surprise for Apple devs. The framework embraces a more declarative and reactive approach to building user interfaces—a complete paradigm shift from interface builder and storyboards. In a way, SwiftUI brings the joys of Swift to UI-building, signaling a coming departure of Objective-C inspired systems.

SwiftUI 1.0 (unofficial versioning), though, proved to still be somewhat a prototype framework, showing signs of its infancy. There were:

Continue reading What’s New in SwiftUI 2.0

Frontier Design’s Video Star is a freemium video editing and creation app with a rich, deep toolkit that allows users to become the stars of their own music videos. From transitions, to blur and color effects, to adding 3D objects and text, and beyond, Video Star has a seemingly endless trove of fun capabilities to explore.

Continue reading Video Star: Creating custom artistic styles for music videos with mobile machine learning

Vertex animation is the keyframed animation of individual vertices of a mesh. It is very beneficial for very complicated motion like that of cloth, hair, or water. It uses a flex controller bone to move the model through a series of flexes — one for each frame.

Vertex animation is included in almost all video or computer-based games. To animate the character, one just needs to vary the sampled line based on the current time.

Continue reading Vertex Animation in Lens Studio for Snapchat Lenses

Ever wondered what the 27th letter in the English alphabet might look like? Or how your appearance would be twenty years from now? Or perhaps how that super-grumpy professor of yours might look with a big, wide smile on his face?

Thanks to machine learning, all this is not only possible, but relatively easy to do with the inference of a powerful neural network (rather than hours spent on Photoshop). The neural networks that make this possible are termed adversarial networks. Often described as one of the coolest concepts in machine learning, they are actually a set of more than one network (usually two) which are continually competing with each other (hence, adversarially), producing some interesting results along the way.

Continue reading StyleGAN: Use machine learning to generate and customize realistic images

ML Kit by google is yet another very efficient and clean way of integrating basic machine learning into your mobile apps. It offers some easy to use prebuilt base APIs:

The best part is that you can use these services on-device (without needing any internet connection for external server calls) or on-cloud (wider training base requiring server calls). Most importantly, this can be integrated in both iOS and Android apps.

Continue reading Text Recognition and Translation on iOS Using ML Kit and Google Translate