For the rotary behavior, it's got nothing to do with the QuNeo and everything to do with Ableton. The way an endless encoder (a "rotary") works is that when going one direction it sends a series of values of 127 and when traveling the other direction is sends a series of values of 1. If the thing you're mapping to just responds as though these are "absolute" values then it will appear to just go between on and off.
When mapping an endless encoder you need to tell Ableton to interpret the data properly. There is a menu to do this that is overlooked very, very easily. It's in the bottom-left corner of Live when mapping something. You generally need to make the mapping first, then it should show up. I've also had to click away from the control I was mapping and then click back to it again in order for it to appear — it's rather finicky. Anyway, just set it to "Relative (lin 2's Comp)" and it should be good to go. I attached a screenshot of what the menu looks like.
For the QuNeo/Editor stuff, I'll clear up some potential misconceptions with edited/custom presets:
When you modify and save a custom preset it gets saved in a "presets" folder in the package contents of the editor itself — there aren't any external presets/folders happening. When you "export" a preset, then it's saved externally. Exported presets are generally for backup purposes, transferring presets to another computer, sharing presets with a friend, or just for storage when you have more than 16 presets you want to keep. Once imported to the editor, they exist inside the editor. QuNeo Editor does not modify any presets in an external presets folder.
To view the real presets folder on Mac you right-click the editor and select "Show Package Contents", then navigate to Contents --> Resources --> presets. On Windows the presets folder is on the same level as the executable (visible and out in the open).
Not sure what you mean by "to get certain control options to customize". You can completely clear a preset (go up to the Edit menu and select "Clear Current Preset") and all options will still be there.
- The very hidden way to set up an endless encoder in Ableton
2, 3, 5, 13, 89, 233, 1597, 28657, 514229, 433494437, 2971215073