Yes, does look like it!! When you think that the book 'For Your Eyes Only' was published in 1960, View to a Kill was a short story within the book; his brother Peter had 'Invasion 1940' published in 1958. Chapter 18' pp 245-251 has details of Auxiliary Units in abundance, maybe they met during their writings and cross pollinated the information??
There is a reference to Ian turning up at the Garth dressed as a German and knocking on the door and asking for Peter. This is not true he turned up with the uniform in a suitcase and only Peter saw it because it was part of Operation Ruthless which was a secret. Also thanks to Ian, Peter and the men at the Garth got a delivery of nice soft navel towels!!
Ian Fleming would certainly have picked up some information on the Auxunits from his brother, but Ian Fleming himself set up his own Special Forces unit drawn mainly from the Navy and Royal Marines. It was named 30 Assault Unit, was tasked mainly with capturing enemy documents and technology and Fleming called them his 'Red Indians.'
30AU has sence been revived as an elite sub-unit of the Royal Marines and is based, if I remember rightly, at Stonehouse Barracks in Plymouth.