All is explained here : Git can't watch all the files and detect changes between the Working Directory and the Staging Area. The node_modules directory is not ignored as expected.

I am using Ubuntu 20.04. Increasing the limit max_user_watches does not fix it for me (+ I have only 4GB of RAM).

Solution : putting the config from the default config file to the user one :

    "files.watcherExclude": {
        "**/.git/objects/**": true,
        "**/.git/subtree-cache/**": true,
        "**/node_modules/**": true,
        "**/.hg/store/**": true
    },

from the defaultSettings.json (Command palette : Preferences : Open default Settings (JSON)) to the userSettings.json (Command palette : Preferences : Open Settings (JSON)). WTF ?! But cool, it works 😎