You can't (without unsafe casts). You can only read from them.
The problem is that you don't know what exactly the list is a list of. It could be a list of any subclass of Number, so when you try to put an element into it, you don't know that the element actually fits into the list.
For example the List might be a list of Bytes, so it would be an error to put a Float into it.