DataHeroes is a Python library and framework that you can use to substantially reduce the size of your dataset, by building a data subset that maintains the statistical properties and corner cases of the full dataset.
You can use this data subset for all your data science operations, such as cleaning, labeling, training and model updates, to build a better ML model and to significantly reduce the time, costs and efforts required to build it.
Build a Better ML Model. 10x Faster.
Use the DataHeroes Python library to reduce your dataset to a small subset that maintains the statistical properties and corner cases of your full dataset and use standard libraries to explore, clean, label, train and tune your models on the smaller subset, and build a higher quality model, faster.
Reduce Dataset Size without Losing Accuracy
Using algorithms from computational geometry known as Coresets, the library computes the Importance of each instance in your dataset and builds a weighted subset that is orders of magnitude smaller than the original dataset yet maintains all its statistical properties and corner cases (this subset is referred to as a Coreset). Save significant time and compute resources without losing accuracy by performing all data science operations on the Coreset.
Build a Better Model (by Systematically Finding and Fixing Errors in Your Data)
Every model is only as good as the quality of the data used to train it. But finding errors in a large dataset is like finding a needle in a haystack. The DataHeroes framework uses the Coreset attributes to systematically identify potential errors and anomalies and flag them for review. Fix the errors and see your model update in real time as the model is re-trained on the Coreset.
Save Time, Money & CO2 (by Training Your Model on the Coreset)
Use Sklearn, Pytorch or other standard libraries to train your model orders of magnitude faster on the Coreset or run many more hyperparameter tuning iterations to improve model quality without requiring excessive compute resources.
Avoid Data Drift (by Continuously Updating Your Model on the Coreset)
Data drift is a common issue when moving models to production, yet always keeping your model up to date by continuously re-training as new data is collected is expensive and time consuming. Our unique Coreset tree structure allows you to add new data and update the Coreset on-the-go, and re-train your model on the Coreset in near real-time.