Skin tags are simply fleshy skin-coloured growths made up of collagen fibres and blood vessels that hang off the skin. Yes, they can be classified as tumors, or odd growths that serve no purpose in the body. However, skin tags in particular are benign tumors, unlike cancerous or malignant counterparts.
That being said, skin tags can become irritated or infected if they keep rubbing against other surfaces. They may also be a nuisance if they frequently catch on jewelry or clothes. Removal is often for the sake of convenience or aesthetics.
You should only really worry if a skin tag grows, changes color, or starts to bleed and itch. Regardless, you can always consult a dermatologist about new growths on your skin to be sure.