I wouldn't take anything off the arm, have the coupling machined a couple of mm. You need all the steel you have on the swing arm. Did you have any difficulty in tightening up the bearing against the crushable distance piece, I found it very difficult to tighten required a long lever and two people to get it tightened.
Ralph
