Yes, unless there is heavy cloud cover you can see the volcano from Reykjavik as long as it is going off.
However, what you can mostly see is the smoke plume. At night time this can be bright orange in color, and reflect in the waters of Faxafloi, the great bay outside Reykjavik. If you come on a Northern Lights cruise on our superyacht you can frequently see that.
How do I see the volcano closer to?
If you want to see the lava then you will have to go to the volcano. Luckily that is less than an hour’s drive. If you don’t have a car it is not a problem, there are many tours available, including one from our friends at Iceland Discover. Or you can get the local bus to the nearest town of Grindavik and walk from there.