Tacit knowledge in skill acquisition is dark knowledge in machine learning

In machine learning, dark knowledge refers to a trained model’s intuition being captured in expressed uncertainty. It seems that using this expert model intuition as the object of optimization is extremely effective in training a new model on the task. The expert-related error signal provides a rich constraint which forces the new model into internalizing the expert’s intuition, just like in tacit knowledge. However, one might argue that machine learning models have access to the very “thoughts” of the expert during distillation.

Backlinks