What will be cleaner and easier to maintain?
All things being equal, probably JavaFX - the API is much more consistent across components. However, this depends much more on how the code is written rather than what library is used to write it.
And what will be faster to build from scratch?
Highly dependent on what you're building. Swing has more components around for it (3rd party as well as built in) and not all of them have made their way to the newer JavaFX platform yet, so there may be a certain amount of re-inventing the wheel if you need something a bit custom. On the other hand, if you want to do transitions / animations / video stuff then this is orders of magnitude easier in FX.
One other thing to bear in mind is (perhaps) look and feel. If you absolutely must have the default system look and feel, then JavaFX (at present) can't provide this. Not a big must have for me (I prefer the default FX look anyway) but I'm aware some policies mandate a restriction to system styles.
Personally, I see JavaFX as the "up and coming" UI library that's not quite there yet (but more than usable), and Swing as the borderline-legacy UI library that's fully featured and supported for the moment, but probably won't be so much in the years to come (and therefore chances are FX will overtake it at some point.)