It is doable as a day trip. It's more efficient to stay overnight because it takes 2 1/2 hours to drive each way but it is doable. If you're going all the way to Montreal, I'd recommend doing it as it's a much different city compared to Montreal. It will be a long day so plan accordingly. Don't miss Montmorency Falls & Fairmont Le Château Frontenac.
Tony2326 - how many days in total do you have available ... that will be the determining factor. "Worth it" questions are completely subjective ... do some research on the two cities and decide which appeal. Only you can decide if a destination is worth visiting, based on your interests and the amount of time you have. There are lots of resources available to help in your research.
@Tony. To avoid confusion do not hijack other peoples posts.