The Unix Code

~ Collaboration Of Code and Processes

Deleting files with special characters

Written by:

I have a folder named akorg✽. That Unicode character causes headaches for me when software makes incorrect assumptions about the text encoding of my file paths, so I’d like to remove it from the name.

The problem

You’d think this would be easy:

$ mv akorg✽ akorg
mv: cannot move ‘akorg✽’ to a subdirectory of itself, ‘akorg/akorg✽’

but—that’s strange—it thinks a folder called akorg already exists. I’m pretty sure there isn’t one:

$ ls -la
total 699K
drwxr-xr-x 15 ak ak   15 Jun 12 17:34 .
drwxr-xr-x 57 ak ak 4.0K Jun 12 17:35 ..
drwxr-xr-x 11 ak ak   21 Jun 12 16:58 akorg✽
drwxr-xr-x  2 ak ak    2 May 28 20:47 Desktop
...




Still, stat says otherwise:


$ stat akorg
  File: ‘akorg’
  Size: 21          Blocks: 33         IO Block: 1536   directory
Device: 15h/21d Inode: 292128      Links: 11
...



So apparently there is an invisible folder in the way. Whatever, I'll just remove it:


$ rmdir akorg
rmdir: failed to remove ‘akorg’: No such file or directory



Right, then. What in the world is this thing?


What I know so far


    

  • I'm using the stable release of ZFS on Linux. Here's the zpool status and zfs properties.
    • 

  • stat returns the same inode for both akorg and akorg✽. Searching by that inode returns only akorg✽:

    $ find . -maxdepth 1 -inum 292128
./akorg✽

    

    • 

  • More things that don't work on the "invisible" folder:

    $ rm akorg
rm: cannot remove ‘akorg’: Is a directory
$ unlink akorg
unlink: cannot unlink ‘akorg’: Is a directory
$ mv akorg akorg_temp
mv: cannot move ‘akorg’ to ‘akorg_temp’: No such file or directory


    

    • 

  • I get the same results in both Bash 4.2.45 and zsh 5.0.0. In both, tab-completion of ak returns only akorg✽/.
    • 

  • The strace of my initial renaming attempt confirms that I'm typing the names correctly and that the attempt is thwarted by the preexistence of a folder named akorg.
    • 

  • This response to a more explicit renaming attempt is puzzling and scary:

    $ mv --verbose --no-target-directory --no-clobber akorg✽ akorg
removed ‘akorg✽’

    

    I don't understand why it claims to have removed akorg✽, or why it would try. Fortunately, ls and stat akorg akorg✽ reveal that nothing is actually gone. Here's the strace.
    • 

  • To rule out encoding quirks as the cause of this, I've temporarily given akorg✽ an intermediate name:

    $ mv --verbose --no-target-directory --no-clobber akorg✽ bananas
‘akorg✽’ -> ‘bananas’

    

    That worked as expected,
    

    $ ls -la
total 715K
drwxr-xr-x 16 ak ak   16 Jun 13 15:11 .
drwxr-xr-x 57 ak ak 4.0K Jun 13 14:03 ..
drwxr-xr-x 11 ak ak   21 Jun 12 16:58 bananas
drwxr-xr-x  2 ak ak    2 May 28 20:47 Desktop
...


    

    but the "invisible" folder is still present:
    

    $ stat bananas akorg
  File: ‘bananas’
  Size: 21          Blocks: 33         IO Block: 1536   directory
Device: 15h/21d Inode: 292128      Links: 11
...
  File: ‘akorg’
  Size: 21          Blocks: 33         IO Block: 1536   directory
Device: 15h/21d Inode: 292128      Links: 11
...


    

    and mv still behaves strangely when I try to use the name akorg:
    

    $ mv --verbose --no-target-directory --no-clobber bananas akorg
removed ‘bananas’

    

    •

This is weird. (And this "answer" started as a comment ;), became a bit long for it.)

Looking at the strace it looks like there are no hidden characters or the like, else I suspect you would have seen it in e.g. (which should have resulted in -1 ENOENT and not 0 if everything was OK):

stat("akorg", {st_mode=S_IFDIR|0755, st_size=21, ...}) = 0

as you do in:

lstat("akorg342234275", {st_mode=S_IFDIR|0755, st_size=21, ...}) = 0

Came across a mail exchange where one person has the opposite problem. ls list the files, but stat give ENOENT – though that was on FreeBSD.

I do not know much about zfs, but could it be that some sync, snapshot or the like has failed and left a corrupted file table? Did you create/have a directory named akorg that you deleted before you tried the mv?

Do not know if you can get some error descriptions by:

# zpool status -v

One thing to try is check the reverse inode lookup (optionally add yet another d) and check path:

# zdb -dddd <pool-name> <inode>

On a folder named baz:

# zdb -dddd qqq 31
Object  lvl   iblk   dblk  dsize  lsize   %full  type
    31    1    16K    512     1K    512  100.00  ZFS directory
                                    264   bonus  ZFS znode
dnode flags: USED_BYTES USERUSED_ACCOUNTED 
dnode maxblkid: 0
path    /baz
uid     1000
gid     1000
atime   Fri Jun 14 12:39:46 2013
mtime   Fri Jun 14 11:55:33 2013
ctime   Fri Jun 14 11:55:33 2013
crtime  Fri Jun 14 11:55:33 2013
gen 1510
mode    40775
size    2
parent  3
links   2
xattr   0
rdev    0x0000000000000000
microzap: 512 bytes, 0 entries

On a directory named foo holding several subdirectories including one named akorg✽:

Object  lvl   iblk   dblk  dsize  lsize   %full  type
    16    1    16K    512     1K    512  100.00  ZFS directory
                                    264   bonus  ZFS znode
dnode flags: USED_BYTES USERUSED_ACCOUNTED 
dnode maxblkid: 0
path    /foo
uid     1000
gid     1000
atime   Fri Jun 14 13:10:38 2013
mtime   Fri Jun 14 12:13:18 2013
ctime   Fri Jun 14 12:13:18 2013
crtime  Fri Jun 14 11:41:53 2013
gen 1482
mode    40775
size    6
parent  3
links   6
xattr   0
rdev    0x0000000000000000
microzap: 512 bytes, 4 entries

    foo1 = 15 (type: Directory)
    foo2 = 18 (type: Directory)
    foo = 19 (type: Directory)
    akorg✽ = 30 (type: Directory)

The settings you have on zfs get all storage/home-ak-annex for various name mod also looks sane (as far as I can tell) as well as the other properties by reading ZFS Properties:

storage/home-ak-annex  utf8only              off                    -
storage/home-ak-annex  normalization         none                   -
storage/home-ak-annex  casesensitivity       sensitive              -

If you build zfs yourself you can enable debug by ./configure --enable-debug and play with the above including -vvvv, -bbbb etc.

Lastly you could open a new Issue on the git.

As you're using zsh, I recommend tab completion or shell globbing (always echo/ls before removing). If those doesn't work, you have a few options:

Check your filename through a hex dump and use $'...' to input it

% touch akorg✽   
% ls | grep akorg | hexdump -C
00000000  61 6b 6f 72 67 e2 9c bd  0a                       |akorg....|
00000009
% ls $'akorgxe2x9cxbd'  
akorg✽
% rm $'akorgxe2x9cxbd'
% ls
%

Note that the final 0a above is the newline at the end of the input, so we don't want it in our name.

Use ls -i to get the inode number and use find to delete it

% touch akorg✽
% ls -i
6128574 akorg✽
% find . -inum 6128574
akorg✽
% find . -inum 6128574 -delete
% ls
%

You could also find it by name (either way, print first!)

Use ls -b to get the escaped name and enter it with $'...' (This works with nonprinting characters but doesn't seem to affect your unicode one, so this is for others' reference).

% ls
foo?bar
% ls -b         
foorbar
% rm foo$'r'bar
% ls
%

Kevin

I'm guessing that you don't have an invisible directory, but rather that stat misprints the unicode characters so that the file name looks like akorg when on-screen.

Two possible solutions:

  1. Use a shell with tab completion (zsh or bash). At the prompt do: rm -rf akorg<tab>. Zsh at least will complete the file name, escaping shell-special characters as necessary.
  2. A one liner, a bit risky: rm -rf akorg?

You could do #2 by hand, something like: ls -1 > filenames. Use some text editor on the file named "filenames" to delete all lines except the offending directory. Add a prefix of rm -rf to that line. Exit editor. Execute as a shell script: sh filenames

jordanm

Deleting files with special characters
0 votes, 0.00 avg. rating (0% score)

Leave a Reply