More often I've heard these called Chimney Swifts, or just Swifts. They do nest in chimneys, and I've noticed them most right around dusk. I used to see them a lot downtown when I parked on the top of the parking deck across from the Reynolds Plaza Building. They are extremely fast and agile, and many times I've thought they were bats at first. I think they compete with bats, since they seem to be taking out insects as they careen around.
I've never heard of a time period for their erratic flights.
One time viewing them stands out. A couple days after the 9-11 attacks my family was eating at a sidewalk restaurant on 4th street (can't remember the name, but it's where Rana Loca is now). Since there was a no-fly restriction, everyone seemed to be particularly aware of the sky, now devoid of any air traffic. We noticed the swifts, and again weren't sure if they were birds or bats. Seemingly hundreds of them swarming and banking through the purple-shadowed dusk. We watched them all through dinner, though they paid no attention to us at all.
There are also bats that come out in large droves from the Stevens Center. I've only been there twice when this has happened but man it is creepy and breath-taking all at the same time. :)