There are two paths to follow here.
In the USA we tend to just polish the barrels as best we can - typically with a cylinder or brake hone (see images above). There are two types of these - the 'straight' version is designed to keep lines perfectly parallel, while the 'ball' version is more random and meant to clean the surface more than cut the wall to any degree. (I've seen this done with wooden dowels with sand paper stuck into the end to make a custom 'flap wheel.')
In the UK, where barrel wall thickness as well as diameter mater relative to the barrel being "in proof" a different method is used. The barrel is set on a cart - not unlike a skate board and run back and forth on an extended version of the 'straight' hones pictured (like 12 inches long) - this method very much resembles 'back boring' in theory. Obviously this device is more aggressive, and tends to give a higher quality finish - tough to do this on your own... and costs a bit more as well.