16 replies to this topic

[Valik]

Why doesn't FileCreateNTFSLink() support cross-volume hard-links? Windows or more specifically NTFS does. I can use FileCreateNTFSLink() to create a hardlink (file) and then move that file to a different volume and the hardlink is maintained. Does the Windows API impose a limitation in this area? If yes, can FileCreateNTFSLink() be extended to detect an attempt to do cross-volume linking and do the following:

• Create the hardlink in a temporary location on the current volume.
• Move the hardlink via Windows API to the correct location on the new volume.

NOTE: I haven't read the code for FileCreateNTFSLink() so I don't know what API functions are being used or any caveats from it so I'm just posting a request from a simple user standpoint. Also, I only tested once in a simple condition that cross-volume links work when manually moved.

Also, looking over the documentation for FileCreateNTFSLink(), I think it should be mentioned that hardlinks only work with files and that when creating a directory link, it's really a junction point. It should also be mentioned that under certain rare conditions, a junction point will not behave as seamlessly as a hardlink will.

[mikkokh]

Maby one good reason for this is that when file is linkked to another drive, program can't verify that target drive has enought free space, pecause it could not monitor target drive. I think that those NTFS links are desinged only to use from same drive that target file - and in that kind use it works fine.

All that i relly hope, is somekind indigator to explorer icon that would can notice what is NTFS link and what is true folder/file - but that hope does not depend anything to AutoIt. ;-)

[blindwig]

I don't think that it's possible to hard link a file from another volume.
The way I understand it, your file tables contain file entries. Each of these entries contains the file details (name, attributes, date stamps, etc) and also a pointer to the HD sector/cluster where the data for that file is stored. When you hard link a file, you're basically creating a second file entry who's data pointer points to the same location as the "source" file. The thing is, I don't think that volume information is stored in that pointer, since it is assumed that all data for all file would be on the same volue as the file table for that volume.
Now for the linking of directories, I can only assume (and I haven't tested this either) that it is using the NTFS volume-linking or volume-mounting or whatever it's called, where you can mount a remote volume/folder as a local folder. I wouldn't recommend using this because I've worked with it before and found it to be flaky.

[Valik]

I obviously must have had the files on the same volume when I tested this previously. I just tried again and it doesn't work. I wish I could recall what I did the first time.

blindwig, you can do more than assume that directories are using anything, I mentioned in my original post that directory linking uses an NTFS junction point.

[blindwig]

I obviously must have had the files on the same volume when I tested this previously. I just tried again and it doesn't work. I wish I could recall what I did the first time.

If you find a solution that works, please post it here. About a month ago I was working on converting a bunch of linux scripts over to Windows, and I got stuck at some scripts that were linking files from another volume (actually an NFS volume on another machine altogether). I never found a Windows equilivant command for this.

blindwig, you can do more than assume that directories are using anything, I mentioned in my original post that directory linking uses an NTFS junction point.

OK, when I did directory linking, I was using the MOUNTVOL and DISKPART commands under Windows XP. According to this page (MS Q205524) the MOUNTVOL command does use NTFS junction points, so yes that's what I was using before.
But again I warn you that they are very flaky. Here's an easy way to crash Window Explorer every time:
Create 2 volumes
Mount a folder from volume 2 into a folder on volume 1
start exploring volume 1 and navigate until you are looking at the folder that is linked to volume 2
delete a file out of this folder
exlporer will try to move the file that is on volume 2 to the recycle bin on volume 1. It will fail. It will hang for a while as it keeps retrying the failed operation, until eventually it just crashes.
It's quirky stuff like that that makes me warn people not to use junction points until MS gets all the bugs worked out.

[mikkokh]

I have sometime found one utility that can do that junction: http://www.sysinternals.com/Utilities/Junction.html

I have some little problems with it sometimes, but it have also sources in share, so i think that it can help...

[blindwig]

I have sometime found one utility that can do that junction: http://www.sysinternals.com/Utilities/Junction.html

I have some little problems with it sometimes, but it have also sources in share, so i think that it can help...

If you're using Windows XP, there are now built-in commands to do the same thing.
Try DiskPart.exe, MountVol.exe, and JoinD.exe
And I believe XP also has the option to do it through the disk manager MSC, but since that's harder to script I haven't really looked into that.

[blindwig]

I found this page that has vb source for file hardlink routines. If anyone can help me translate them to AutoIt, I'd be happy to write wrappers for them.

Here's the code:

Private Declare Function CreateHardLink _
Lib "kernel32" Alias "CreateHardLinkA" _
(ByVal lpFileName As String, _
 ByVal lpExistingFileName As String, _
 lpSecurityAttributes As Any) As Long

p_lngRtn = CreateHardLink(p_strFileName, _
p_strExistingFileName, ByVal 0&)

and

Private Declare Function GetFileInformationByHandle _
Lib "kernel32" _
(ByVal hFile As Long, _
 lpFileInformation As BY_HANDLE_FILE_INFORMATION) As Long

p_lngFileHwnd = CreateFile(lpFileName:=p_strFileName, _
dwDesiredAccess:=GENERIC_ALL, _
dwShareMode:=FILE_SHARE_WRITE And _
FILE_SHARE_READ, _
lpSecurityAttributes:=0&, _
dwCreationDisposition:=OPEN_EXISTING, _
dwFlagsAndAttributes:=0&, _
hTemplateFile:=0&)
If p_lngFileHwnd <> INVALID_HANDLE_VALUE Then
p_lngRtn = GetFileInformationByHandle(hFile:=p_lngFileHwnd, _
lpFileInformation:=p_typFileInfo)
If p_lngRtn <> 0 Then
MsgBox "Number of Hard Links: " _
& p_typFileInfo.nNumberOfLinks

[blindwig]

OK, using All API to lookup the CreateHardLink function in kernel32, I found this VB wrapper source:

Plain Text        expandcollapse  popup

'This example only works on NTFS !!!
Private Declare Function CreateHardLink Lib "kernel32.dll" Alias "CreateHardLinkA" (ByVal lpFileName As String, ByVal lpExistingFileName As String, ByRef lpSecurityAttributes As Any) As Long
Private Sub Form_Load()
    'KPD-Team 2000
    'URL: http://www.allapi.net/
    'E-Mail: KPDTeam@allapi.net
    Dim sSave As String
    'create a file and write something in it
    Open "c:\test1.tst" For Output As #1
        Print #1, "Hello ";
    Close #1
    'create a hard link (a second filename that points to the same file)
    CreateHardLink "c:\test2.tst", "c:\test1.tst", ByVal 0&
    'open the second filename and write something in the file
    Open "c:\test2.tst" For Append As #1
        Print #1, "World"
    Close #1
    'create a string buffer
    sSave = String(FileLen("c:\test2.tst"), 0)
    'read the contents of the first file
    Open "c:\test1.tst" For Binary As #1
        Get #1, , sSave
    Close #1
    'show the result. It should be 'Hello World'
    MsgBox sSave
    'kill the two filenames
    Kill "c:\test1.tst"
    Kill "c:\test2.tst"
End Sub

Looks pretty straight forward. The only thing I don't know how to do is that last parameter, the "ByVal 0&". I assume that's a null-pointer? How to I signify that to DllCall()? Should I use "'ptr', 0" or something like that?

[blindwig]

OK, I threw this together:

Func _NtfsCreateHardlink($sDestination, $sSource, $vKernel32 = 'Kernel32.dll')
    Local $result = DllCall($vKernel32, 'long', 'CreateHardLink', 'string', $sDestination, 'string', $sSource, 'ptr', 0)
    SetError(@Error)
    Return $result
EndFunc

and tested it:

_NtfsCreateHardlink('C:\test1.txt','c:\boot.ini')

$dll_Kernel32 = DllOpen('kernel32.dll')
_NtfsCreateHardlink('C:\test2.txt','c:\boot.ini',$dll_Kernel32)
DllClose($dll_Kernel32)

[Valik]

Why are you duplicating the function that is implemented in AutoIt?

[blindwig]

AutoIt built-in function will add a hardlink, but there is no function to determine if a file is already hardlinked.

I'm writing a script that will hard link a bunch of files from 1 directory into another directory, and then add files into the second directory, and then remove all the hardlinked files in that second directory.

I want to do this with hard linking instead of just copying and then deleting the files because I think it would be faster and not consume as much system resources.

So translating the first function into AutoIt was just an exercise for me - yes I know that AutoIt already has a hardlinking function. What I really need is the second function, the one that call tell you if a file is hardlinked already. This is my first attempt at DLL calls from AutoIt and while the first one seemed rather straight-forward, the second one seems more difficult.

[MHz]

What I really need is the second function, the one that call tell you if a file is hardlinked already.

Can I ask why you need to know if a file is hardlinked?
I normally treat them as a copy of the real file. The only difference is whether it is stored on the drive physically again or the Master File Table reference to the same file. The version of a file should be the giveaway to an updated file no matter which concept is used.
Whether you copy or hardlink a file, then you still get the same options with deleting etc them both. Hardlinks are a smarter spacesaving method to choose within the same partition/volume, including advantages you state previous.
You may need an external library to do this as a DllCall(). References to hardlinks have been known to be saved in the registry of even in the file streams.

[Valik]

Can I ask why you need to know if a file is hardlinked?
I normally treat them as a copy of the real file. The only difference is whether it is stored on the drive physically again or the Master File Table reference to the same file. The version of a file should be the giveaway to an updated file no matter which concept is used.
Whether you copy or hardlink a file, then you still get the same options with deleting etc them both. Hardlinks are a smarter spacesaving method to choose within the same partition/volume, including advantages you state previous.
You may need an external library to do this as a DllCall(). References to hardlinks have been known to be saved in the registry of even in the file streams.

That's not true as far as I know. The number of hard-links for a file is stored in the MFT. Any other data concerning a hard-link is application specific and completely irrelevant to Windows. Some applications may store hard-link information in the registry or streams so that they can show the user a list of which files are linked to each other. However, this is application-specific. They are storing this information by choice because there is no way to retrieve that information from Windows without crawling the entire volume. The information stored in the registry or stream can be deleted and you will lose application functionality for the hard-link but you will not lose the hard-link itself.

The GetFileInformationByHandle() is available on all 32-bit versions of Windows. It's in kernel32.dll, no 3rd-party DLLs are required. The structure has a field which is the number of links to a file. That's all the information that can be obtained for a hard-link.

One good reason to know if a file is hard-linked or not is for updating it. If you try to move a file and overwrite a hard-link of the same name, the link is destroyed. However, you can safely copy a file and overwrite a hard-link and this will correctly update the hard-link. This seems unintuitive to me as a user but from the file-system's perspective it makes sense since a move just redirects the MFT entry instead of physically moving the bits on the disk.

[blindwig]

Can I ask why you need to know if a file is hardlinked?

I am working on converting some scripts written for linux over to a Windows environment. Linux has symbolic links that make it easy to make files appear in different folder and/or have different names. Windows symbolic links (AKA shortcut files) are a lot less useful because neither the kernel nor the file system de-references them automatically, so it is up to the application to recognize a shortcut and follow it to the source.
So I'm trying to come up with a work-around using hardlinks. The first limitation I'm facing is that you can't hardlink across volumes (meaning that other partitions, drives, and remote shares are all out). Secondly there is no easy way to know if a file is a hardlink of another file or not. Third, if you do know that a file is a hardlink, you have no way to know what the source of that file is.

I normally treat them as a copy of the real file. The only difference is whether it is stored on the drive physically again or the Master File Table reference to the same file. The version of a file should be the giveaway to an updated file no matter which concept is used.
Whether you copy or hardlink a file, then you still get the same options with deleting etc them both. Hardlinks are a smarter spacesaving method to choose within the same partition/volume, including advantages you state previous.
You may need an external library to do this as a DllCall(). References to hardlinks have been known to be saved in the registry of even in the file streams.

Yes I get all that. I'm using hardlinks to avoid having to copy files. I have a lot of files to copy, and some are up in the 500MB-2GB range, which will take significient time and resources to copy, even if I do it on a RAID system. Hardlinking files around is a much better solution.

I guess I really wish there was a such thing as a virtual filesystem for NTFS. I wonder how hard it would be to write a driver to give you a virtual filesystem in Windows. You couldn't boot from it (unless you re-wrote the NT Loader and all kind of other low-level kernel stuff) but it would still be pretty cool...

[NikLok]

I am working on converting some scripts written for linux over to a Windows environment. Linux has symbolic links that make it easy to make files appear in different folder and/or have different names. Windows symbolic links (AKA shortcut files) are a lot less useful because neither the kernel nor the file system de-references them automatically, so it is up to the application to recognize a shortcut and follow it to the source.

I am also use NTFS link for saving HDD workspace. And I also have big needs feature to determine the type of file - it is real file or link. It is important for me because, i share the directory with NFTS link to the network. But in MS NTFS not link not shared and must shared one by one. My script must check that all catalogs on shared catalog is accessible by the net. I think that need to do the hierahicaly walking tree and heck that all links are shared also. But a can't recognize file from link!!!!

[Valik]

I don't know what your post has to do with this thread other than being loosely based on links. Anyway. Locked.