While smartphone usage become more and more pervasive,people start also asking to which extent such devices can bemaliciously exploited as “tracking devices”. The concern isnot only related to an adversary taking physical or remotecontrol of the device, but also to what a passive adversarywithout the above capabilities can observe from the devicecommunications. Work in this latter direction aimed, forexample, at inferring the apps a user has installed on hisdevice, or identifying the presence of a specific user withina network.In this paper, we move a step forward: we investigate towhich extent it is feasible to identify the specific actions thata user is doing on mobile apps, by eavesdropping their en-crypted network traffic. We design a system that achievesthis goal by using advanced machine learning techniques.We did a complete implementation of this system and run athorough set of experiments, which show that it can achieveaccuracy and precision higher than 95% for most of the con-sidered actions.