Here is a solution:
On Windows, Arduino IDE comes with a tool named listComPorts.exe, which will list all COM ports and their USB info like VID & PID. There is also a file named arduino.inf in drivers folder contains all known device types and their VID&PID etc, and we can look for the connected device.
On *nix, the IDE take advantage of libusb and no such a tool, and it's easy to find a tool to list usb info on these platforms.
This go package mentioned in the question is a good choice, which also uses libusb and could build on all platforms. (On windows, you have to install mingw to build the C code, cygwin & msys don't work, I tried TDM-GCC and it works. You also have to add LDFLAGS: -lsetupapi (in serial.go) to successfully link the code.) After list all serial ports, you can search for 'Arduino' to determine the port. 'CH340' is also a keyword to search as many modified boards use this chipset.
New version of the IDE don't have listComPorts.exe, they all uses libusb. I find this tool from version 1.6.5.