Discussion:
[Mingw-w64-public] How to recognize symlinks in WIN32?
Greg Jung
2015-01-14 17:33:35 UTC
Permalink
Hi Folks,
I've been trying to program a recognition of NTFS symbolic link files;
What I;ve gleaned
from MSDN procedures don't seem to work. Below are coded three methods

1) (untested) open file and call GetFileInformationByHandleEx. I can't get
winbase.h header definitions recognized
2) GetFileAttributes() works ok for Junctions doesn't recognize symlink
files
3) FindFirstEx . same result as 2)

There must be a way, since cygwin 'ls -a' and msys2 'ls -a' commands will
show links.

in the calling routine, I modify result of stat() with system calls:
#ifdef _WIN32
int addlink = 0;
fstat_win32(filename.c_str(), addlink);
int actStat = stat(filename.c_str(), &statStruct);
if( addlink != 0) cout << " addlink came back " << endl;
statStruct.st_mode |= addlink;
#else
int actStat = lstat( testDir.c_str(), &statStruct);
#endif

where fstat_win32 now has a graveyard full of attempts: ( When a directory
link is encountered, however, the test succeeeds.)

void fstat_win32(const string& filepath, int& st_mode)
{
DWORD dwattrib, reparsetag;
DString st;
st=filepath;
#if 1
//
// This method has not been tested: it will not compile -
GetFileInformationByHandleEx
// does not get through from the winbase.h header
//
HANDLE hFile;
BOOL success;
DWORD fattrib[2];
hFile = CreateFile( (TCHAR *)st.c_str(),
FILE_GENERIC_READ,
FILE_SHARE_READ | FILE_SHARE_WRITE,
0,
OPEN_EXISTING,
FILE_FLAG_OPEN_REPARSE_POINT,
0);
if (hFile == INVALID_HANDLE_VALUE) {
CloseHandle(hFile);
cout << " stat_win32: could not open " + st << endl;
return;
}
else {
success = GetFileInformationByHandleEx( hFile,
FileAttributeTagInfo,
&fattrib,
sizeof(fattrib));
dwattrib = fattrib[0];
reparsetag = fattrib[1];
}
CloseHandle(hFile);

// dwattrib = GetFileAttributes( (TCHAR *)st.c_str()); // This doesn't
work to find symlinks
if( (dwattrib & FILE_ATTRIBUTE_REPARSE_POINT) != 0 ) {
st_mode |= S_IFLNK;
cout << " fstat_win32: symbolic link detected and flagged!! " <<
filepath ;
}
else st_mode=0;
#elif 0
DWORD dwlen;
HANDLE hFind;
BOOL foundnext;
WIN32_FIND_DATA FindFileData;
// This doesn't work to find symlinks
hFind = FindFirstFileEx( (TCHAR *) st.c_str(),
FindExInfoStandard,
&FindFileData,
FindExSearchNameMatch, NULL, 0);
if (hFind == INVALID_HANDLE_VALUE)
return;
dwattrib = FindFileData.dwFileAttributes;
if( (dwattrib & FILE_ATTRIBUTE_REPARSE_POINT) != 0 ) {
st_mode |= S_IFLNK;
cout << " fstat_win32: symbolic link detected and flagged!! " <<
filepath ;
dwattrib = FindFileData.dwReserved0;
// IO_REPARSE_TAG_SYMLINK _MOUNT_POINT etc.
}
FindClose(hFind);
#endif
return;
}
Greg Jung
2015-01-14 23:54:49 UTC
Permalink
Hi all,
I've been doing some experimentation today and discovered another twist
in the story
which relates to cygwin - so I forward this on to cygwin-user list, also.
I have been testing the code below against symlinks created
in cygwin64 (I'm running win7 64-bit; I have mingw/msys, msys2 (64), and
cygwin64 installations).
My code hadn't detected symlink files but all of thesymlinks I was trying
were cygwin64.
I created some symlinks natively, using an ln.exe (from
http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html)
and using mklink.exe; these links *-do show up-* as reparse points in my
detection scheme 3), (but 2) should also work) described below.
The links I couldn't detect earlier were created by cygwin install
procedure, i.e. cygwin64/lib/noX has two files and
two symlinks pointing to the files: fstat_win32 below didn't find a
FILE_ATTRIBUTE_REPARSE_POINT in its mask.

Nevertheless, cygwin and msys2 "ls -la" commands show a symlink. They work
like a symlink should.
So the question becomes, "why do cygwin symlinks look different, and how
can a user program detect this attribute?

(I've edited the original message to correct the lstat call )
Post by Greg Jung
Hi Folks,
I've been trying to program a recognition of NTFS symbolic link files;
What I;ve gleaned
from MSDN procedures don't seem to work. Below are coded three methods
1) (untested) open file and call GetFileInformationByHandleEx. I can't
get winbase.h header definitions recognized
2) GetFileAttributes() works ok for Junctions doesn't recognize symlink
files
3) FindFirstEx . same result as 2)
There must be a way, since cygwin 'ls -a' and msys2 'ls -a' commands will
show links.
in the calling routine, I modify result of stat() (from glibc, I take it)
#ifdef _WIN32
int addlink = 0;
fstat_win32(filename.c_str(), addlink);
int actStat = stat(filename.c_str(), &statStruct);
if( addlink != 0) cout << " addlink came back " << endl;
statStruct.st_mode |= addlink;
#else
int actStat = lstat( filename.c_str(), &statStruct);
#endif
where fstat_win32 now has a graveyard full of attempts: ( When a directory
link is encountered, however, the test succeeeds.)
void fstat_win32(const string& filepath, int& st_mode)
{
DWORD dwattrib, reparsetag;
DString st;
st=filepath;
#if 1
//
// This method has not been tested: it will not compile -
GetFileInformationByHandleEx
// does not get through from the winbase.h header
//
HANDLE hFile;
BOOL success;
DWORD fattrib[2];
hFile = CreateFile( (TCHAR *)st.c_str(),
FILE_GENERIC_READ,
FILE_SHARE_READ | FILE_SHARE_WRITE,
0,
OPEN_EXISTING,
FILE_FLAG_OPEN_REPARSE_POINT,
0);
if (hFile == INVALID_HANDLE_VALUE) {
CloseHandle(hFile);
cout << " stat_win32: could not open " + st << endl;
return;
}
else {
success = GetFileInformationByHandleEx( hFile,
FileAttributeTagInfo,
&fattrib,
sizeof(fattrib));
dwattrib = fattrib[0];
reparsetag = fattrib[1];
}
CloseHandle(hFile);
// dwattrib = GetFileAttributes( (TCHAR *)st.c_str()); // This doesn't
work to find symlinks
if( (dwattrib & FILE_ATTRIBUTE_REPARSE_POINT) != 0 ) {
st_mode |= S_IFLNK;
cout << " fstat_win32: symbolic link detected and flagged!! " <<
filepath ;
}
else st_mode=0;
#elif 0
DWORD dwlen;
HANDLE hFind;
BOOL foundnext;
WIN32_FIND_DATA FindFileData;
// This doesn't work to find symlinks
hFind = FindFirstFileEx( (TCHAR *) st.c_str(),
FindExInfoStandard,
&FindFileData,
FindExSearchNameMatch, NULL, 0);
if (hFind == INVALID_HANDLE_VALUE)
return;
dwattrib = FindFileData.dwFileAttributes;
if( (dwattrib & FILE_ATTRIBUTE_REPARSE_POINT) != 0 ) {
st_mode |= S_IFLNK;
cout << " fstat_win32: symbolic link detected and flagged!! " <<
filepath ;
dwattrib = FindFileData.dwReserved0;
// IO_REPARSE_TAG_SYMLINK _MOUNT_POINT etc.
}
FindClose(hFind);
#endif
return;
}
dw
2015-01-15 02:16:25 UTC
Permalink
I'm not quite sure what you are looking for. However, maybe this will help:

I created an NTFS symbolic file link with (data.raw is just a normal file):

mklink c:\idelme\data2.raw c:\idelme\data.raw

From there, I was able to see that it was a symbolic link with:

#include <windows.h>
#include <stdio.h>

void TestMe(const char *name)
{
printf("name: %s\n", name);
DWORD d = GetFileAttributes(name);
printf("%x %d\n", d, d & FILE_ATTRIBUTE_REPARSE_POINT);

WIN32_FIND_DATA ffd;
HANDLE h = FindFirstFile(name, &ffd);

printf("%x\n\n", ffd.dwReserved0);
CloseHandle(h);
}

int main()
{
TestMe("C:\\idelme\\data.raw");
TestMe("C:\\idelme\\data2.raw");
}

Data.raw does not have the FILE_ATTRIBUTE_REPARSE_POINT attribute set.
Similarly, ffd.dwReserved0 is 0.
Data2.raw DOES have the FILE_ATTRIBUTE_REPARSE_POINT attribute. And
ffd.dwReserved0 shows IO_REPARSE_TAG_SYMLINK.

FWIW.

dw
Post by Greg Jung
Hi Folks,
I've been trying to program a recognition of NTFS symbolic link
files; What I;ve gleaned
from MSDN procedures don't seem to work. Below are coded three methods
1) (untested) open file and call GetFileInformationByHandleEx. I
can't get winbase.h header definitions recognized
2) GetFileAttributes() works ok for Junctions doesn't recognize
symlink files
3) FindFirstEx . same result as 2)
There must be a way, since cygwin 'ls -a' and msys2 'ls -a' commands
will show links.
#ifdef _WIN32
int addlink = 0;
fstat_win32(filename.c_str(), addlink);
int actStat = stat(filename.c_str(), &statStruct);
if( addlink != 0) cout << " addlink came back " << endl;
statStruct.st_mode |= addlink;
#else
int actStat = lstat( testDir.c_str(), &statStruct);
#endif
where fstat_win32 now has a graveyard full of attempts: ( When a
directory link is encountered, however, the test succeeeds.)
void fstat_win32(const string& filepath, int& st_mode)
{
DWORD dwattrib, reparsetag;
DString st;
st=filepath;
#if 1
//
// This method has not been tested: it will not compile -
GetFileInformationByHandleEx
// does not get through from the winbase.h header
//
HANDLEhFile;
BOOL success;
DWORD fattrib[2];
hFile = CreateFile( (TCHAR *)st.c_str(),
FILE_GENERIC_READ,
FILE_SHARE_READ | FILE_SHARE_WRITE,
0,
OPEN_EXISTING,
FILE_FLAG_OPEN_REPARSE_POINT,
0);
if (hFile == INVALID_HANDLE_VALUE) {
CloseHandle(hFile);
cout << " stat_win32: could not open " + st << endl;
return;
}
else {
success = GetFileInformationByHandleEx(hFile,
FileAttributeTagInfo,
&fattrib,
sizeof(fattrib));
dwattrib = fattrib[0];
reparsetag = fattrib[1];
}
CloseHandle(hFile);
// dwattrib = GetFileAttributes( (TCHAR *)st.c_str()); // This
doesn't work to find symlinks
if( (dwattrib & FILE_ATTRIBUTE_REPARSE_POINT) != 0 ) {
st_mode |= S_IFLNK;
cout << " fstat_win32: symbolic link detected and flagged!! "
<< filepath ;
}
else st_mode=0;
#elif 0
DWORD dwlen;
HANDLEhFind;
BOOLfoundnext;
WIN32_FIND_DATA FindFileData;
// This doesn't work to find symlinks
hFind = FindFirstFileEx( (TCHAR *) st.c_str(),
FindExInfoStandard,
&FindFileData,
FindExSearchNameMatch, NULL, 0);
if (hFind == INVALID_HANDLE_VALUE)
return;
dwattrib = FindFileData.dwFileAttributes;
if( (dwattrib & FILE_ATTRIBUTE_REPARSE_POINT) != 0 ) {
st_mode |= S_IFLNK;
cout << " fstat_win32: symbolic link detected and flagged!! "
<< filepath ;
dwattrib = FindFileData.dwReserved0;
// IO_REPARSE_TAG_SYMLINK _MOUNT_POINT etc.
}
FindClose(hFind);
#endif
return;
}
------------------------------------------------------------------------------
New Year. New Location. New Benefits. New Data Center in Ashburn, VA.
GigeNET is offering a free month of service with a new server in Ashburn.
Choose from 2 high performing configs, both with 100TB of bandwidth.
Higher redundancy.Lower latency.Increased capacity.Completely compliant.
http://p.sf.net/sfu/gigenet
_______________________________________________
Mingw-w64-public mailing list
https://lists.sourceforge.net/lists/listinfo/mingw-w64-public
Greg Jung
2015-01-15 03:11:44 UTC
Permalink
Yes I've seen that, if my second post appeared, the symlinks created with
cygwin are the ones giving me trouble. These links are invisible to
CMD.exe, by someone's
design:

CYGWIN- created links in
Directory of e:\cygwin64\lib\nox

03/31/2014 09:39 AM <DIR> .
03/31/2014 09:39 AM <DIR> ..
07/01/2013 03:24 AM 336,710 libXpm-noX.a
07/01/2013 03:24 AM 43,690 libXpm-noX.dll.a
2 File(s) 380,400 bytes
2 Dir(s) 85,657,726,976 bytes free

from mingw.org/msys bash:
***@Homerw7 /f/git/bldgdl
$ ls -la /e/cygwin64/lib/nox
total 634
drwxr-xr-x 2 greg Administrators 4096 Mar 31 2014 .
drwxr-xr-x 77 greg Administrators 262144 Sep 25 21:36 ..
-rw-r--r-- 1 greg Administrators 336710 Jul 1 2013 libXpm-noX.a
-rw-r--r-- 1 greg Administrators 43690 Jul 1 2013 libXpm-noX.dll.a
-rw-r--r-- 1 greg Administrators 38 Mar 29 2014 libXpm.a
-rw-r--r-- 1 greg Administrators 46 Mar 29 2014 libXpm.dll.a

from cygwin/bash:
$ ls -la /e/cygwin64/lib/nox
total 638
drwxrwxrwx+ 1 greg None 0 Mar 31 2014 .
drwxrwxrwx+ 1 greg None 0 Sep 25 21:36 ..
lrwxrwxrwx 1 greg None 12 Mar 29 2014 libXpm.a -> libXpm-noX.a
lrwxrwxrwx 1 greg None 16 Mar 29 2014 libXpm.dll.a -> libXpm-noX.dll.a
-rwxrwxrwx+ 1 greg None 336710 Jul 1 2013 libXpm-noX.a
-rwxrwxrwx+ 1 greg None 43690 Jul 1 2013 libXpm-noX.dll.a

And the same result from msys2 bash.
The CMD.exe directory command of a native symlink looks like so:
Directory of c:\msys64\opt

01/14/2015 03:02 PM <DIR> .
01/14/2015 03:02 PM <DIR> ..
12/08/2014 09:40 PM <DIR> bin
11/08/2014 02:47 PM <DIR> doc
11/06/2014 01:29 AM <DIR> etc
11/17/2014 06:22 PM <DIR> include
11/04/2014 01:09 AM <DIR> info
12/08/2014 04:37 PM <DIR> lib
12/08/2014 09:39 PM <DIR> lib64
05/24/2014 09:11 AM <DIR> man
11/18/2014 04:23 PM <DIR> pkgconfig
11/05/2014 09:16 PM <DIR> share
11/04/2014 01:09 AM <DIR> var
01/14/2015 03:02 PM <SYMLINK> ww [yypkg.exe]
11/08/2014 08:57 AM <DIR> x86_64-w64-mingw32
01/14/2015 02:36 PM <SYMLINK> yy [yypkg.exe]
11/04/2014 01:04 AM 2,348,046 yypkg.exe
3 File(s) 2,348,046 bytes
14 Dir(s) 40,575,012,864 bytes free
on cygwin the native links ww,yy look just like cygwin symlinks:
$ls -la
total 2576
drwxrwxrwx+ 1 greg None 0 Jan 14 15:02 .
drwxrwxrwx+ 1 greg None 0 Nov 17 18:21 ..
drwxrwxrwx+ 1 greg None 0 Dec 8 21:40 bin
drwxrwxrwx+ 1 greg None 0 Nov 8 14:47 doc
drwxrwxrwx+ 1 greg None 0 Nov 6 01:29 etc
drwxrwxrwx+ 1 greg None 0 Nov 17 18:22 include
drwxrwxrwx+ 1 greg None 0 Nov 4 01:09 info
drwxrwxrwx+ 1 greg None 0 Dec 8 16:37 lib
drwxrwxrwx+ 1 greg None 0 Dec 8 21:39 lib64
drwxrwxrwx+ 1 greg None 0 May 24 2014 man
drwxrwxrwx+ 1 greg None 0 Nov 18 16:23 pkgconfig
drwxrwxrwx+ 1 greg None 0 Nov 5 21:16 share
drwxrwxrwx+ 1 greg None 0 Nov 4 01:09 var
lrwxrwxrwx 1 Administrators None 9 Jan 14 15:02 ww -> yypkg.exe
drwxrwxrwx+ 1 greg None 0 Nov 8 08:57 x86_64-w64-mingw32
lrwxrwxrwx 1 Administrators None 9 Jan 14 14:36 yy -> yypkg.exe
-rwxrwxrwx+ 1 greg None 2348046 Nov 4 01:04 yypkg.exe
Post by dw
Data.raw does not have the FILE_ATTRIBUTE_REPARSE_POINT attribute set.
Similarly, ffd.dwReserved0 is 0.
Data2.raw DOES have the FILE_ATTRIBUTE_REPARSE_POINT attribute. And
ffd.dwReserved0 shows IO_REPARSE_TAG_SYMLINK.
FWIW.
dw
For some reason my second post was bounced (maybe it didn't like a link I
referenced)

The links I couldn't detect earlier were created by cygwin install
Post by dw
procedure, i.e. cygwin64/lib/noX has two files and
two symlinks pointing to the files: fstat_win32 below didn't find a
FILE_ATTRIBUTE_REPARSE_POINT in its mask.
Nevertheless, cygwin and msys2 "ls -la" commands show a symlink. They
work like a symlink should.
So the question becomes, "why do cygwin symlinks look different, and how
can a user program detect this attribute?
Hi Folks,
I've been trying to program a recognition of NTFS symbolic link files;
What I;ve gleaned
from MSDN procedures don't seem to work. Below are coded three methods
1) (untested) open file and call GetFileInformationByHandleEx. I can't
get winbase.h header definitions recognized
2) GetFileAttributes() works ok for Junctions doesn't recognize symlink
files
3) FindFirstEx . same result as 2)
David Macek
2015-01-15 08:23:57 UTC
Permalink
Yes I've seen that, if my second post appeared, the symlinks created with cygwin are the ones giving me trouble. These links are invisible to CMD.exe, by someone's
CYGWIN- created links in
Directory of e:\cygwin64\lib\nox
03/31/2014 09:39 AM <DIR> .
03/31/2014 09:39 AM <DIR> ..
07/01/2013 03:24 AM 336,710 libXpm-noX.a
07/01/2013 03:24 AM 43,690 libXpm-noX.dll.a
2 File(s) 380,400 bytes
2 Dir(s) 85,657,726,976 bytes free
I think these are cygwin emulated symlinks:

They are visible, just not by default. I suspect they are marked with the system attribute. Use "dir /as" to show them. You should see a small size (in order of tens of bytes).

You can instruct cygwin to create native NTFS symlinks, but due to a different design, there are some restrictions. See this: <https://cygwin.com/cygwin-ug-net/using.html#pathnames-symlinks>
So the question becomes, "why do cygwin symlinks look different, and how can a user program detect this attribute?
I assume you could detect them using cygwin *stat calls. Maybe by compiling against cygwin headers and cygwin1.dll, or maybe by extracting the relevant code from cygwin sources (you'd have to check the relevant licenses).
--
David Macek
Greg Jung
2015-01-15 20:39:54 UTC
Permalink
Thanks all, let me summarize the situation:

cygwin has its own trick to make a symlnk where the native OS doesn't
cooperate, which is a simple file beginning with the char[11]='!<symlink>'
then 0xFF,0xFE, then wide-char name of where the link points.
In windows a symlink creator needs elevated privilege and so native
symlinks may not be sufficient.

cygwin lstat is trained to recognize this. The first clue is that it is
a system file - if the system attribute is erased, cygwin looks at a simple
file.
(It doesn't care that it is marked hidden or not). Then symlinks would have
'!<symlink>',0xff,0xfe as first 12 bytes;

incorporating winsymlinks:native invokes the cygwin "ln" command to
create native symlinks if the shell is run as administrator, cywin links if
not.
winsymlinks:nativestrict would break non-native symlink creation.

That's all very useful thanks for the help.
Post by Greg Jung
Yes I've seen that, if my second post appeared, the symlinks created
with cygwin are the ones giving me trouble. These links are invisible to
CMD.exe, by someone's
Post by Greg Jung
CYGWIN- created links in
Directory of e:\cygwin64\lib\nox
03/31/2014 09:39 AM <DIR> .
03/31/2014 09:39 AM <DIR> ..
07/01/2013 03:24 AM 336,710 libXpm-noX.a
07/01/2013 03:24 AM 43,690 libXpm-noX.dll.a
2 File(s) 380,400 bytes
2 Dir(s) 85,657,726,976 bytes free
They are visible, just not by default. I suspect they are marked with the
system attribute. Use "dir /as" to show them. You should see a small size
(in order of tens of bytes).
You can instruct cygwin to create native NTFS symlinks, but due to a
different design, there are some restrictions. See this: <
https://cygwin.com/cygwin-ug-net/using.html#pathnames-symlinks>
Post by Greg Jung
So the question becomes, "why do cygwin symlinks look different, and how
can a user program detect this attribute?
I assume you could detect them using cygwin *stat calls. Maybe by
compiling against cygwin headers and cygwin1.dll, or maybe by extracting
the relevant code from cygwin sources (you'd have to check the relevant
licenses).
--
David Macek
Adrien Nader
2015-01-15 22:09:54 UTC
Permalink
Post by Greg Jung
cygwin has its own trick to make a symlnk where the native OS doesn't
cooperate, which is a simple file beginning with the char[11]='!<symlink>'
then 0xFF,0xFE, then wide-char name of where the link points.
In windows a symlink creator needs elevated privilege and so native
symlinks may not be sufficient.
cygwin lstat is trained to recognize this. The first clue is that it is
a system file - if the system attribute is erased, cygwin looks at a simple
file.
(It doesn't care that it is marked hidden or not). Then symlinks would have
'!<symlink>',0xff,0xfe as first 12 bytes;
incorporating winsymlinks:native invokes the cygwin "ln" command to
create native symlinks if the shell is run as administrator, cywin links if
not.
winsymlinks:nativestrict would break non-native symlink creation.
That's all very useful thanks for the help.
In a few words: cygwin, and therefore msys*, are additional layers which
provide almost posix on top of win32 and do their very own stuff.

Win32 and POSIX symlinks have little in common besides the name and you
shouldn't have to take special care for files which are created by
either since they're emulating the functionality, even on systems
without reparse points, junctions and win32 symlinks (i.e. they use
dirty tricks to do it).

(and actually I think msys doesn't symlink but copies)
--
Adrien Nader
Loading...