By default, you can’t add labels in the workplace switcher in Ubuntu.
To enable them, go to System > Preferences > Appearance.
Click the Visual Effects tab.
Then, click None, to disable all effects.
Now, you should be able to right-click your workplace switcher, click Preferences, and add labels to each workspace.