Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

mkdir function #22

Open
timchurchard opened this issue Mar 12, 2025 · 5 comments · May be fixed by #24
Open

mkdir function #22

timchurchard opened this issue Mar 12, 2025 · 5 comments · May be fixed by #24
Assignees
@adamdecaf
Labels
enhancement New feature or request

Comments

@timchurchard
Copy link

Please could the UploadFile function create directories. Or could you expose the FTP mkdir
@adamdecaf
Copy link
Member

We probably can, but with the underlying libraries need to chdir it might be unable to fully resolve some relative paths to mkdir.

@adamdecaf adamdecaf added the enhancement New feature or request label Mar 12, 2025
@adamdecaf adamdecaf self-assigned this Mar 12, 2025
adamdecaf added a commit that referenced this issue Mar 12, 2025
@adamdecaf
feat: add CreateUploadDirectories to mkdir inside UploadFile
72224b6 
Fixes: #22
@adamdecaf
Copy link
Member

Can you try out the branch before we merge/release? Some servers/permission setups don't allow mkdir so we have to hide it behind a config option (and to keep the behavior stable).

go get github.com/moov-io/go-ftp@feat-upload-can-mkdir

@adamdecaf adamdecaf linked a pull request Mar 12, 2025 that will close this issue
Draft
adamdecaf added a commit that referenced this issue Mar 12, 2025
@adamdecaf
feat: add CreateUploadDirectories to mkdir inside UploadFile
d09ec89 
Fixes: #22
@timchurchard
Copy link
Author

timchurchard commented Mar 13, 2025 •
edited
Loading

Hi, thank you for working on the issues I raised so quickly. I've tested the feat-upload-can-mkdir but I'm getting "FTP: mkdir 2025/03/13/ (from /) failed: 550 Create directory operation failed." I've been asked to send files to FTP partitioned by date as above, and I'm testing with this docker compose. If I connect from the shell to this container I am able to mkdir and chdir.

vsftpd:
    image: fauria/vsftpd:latest
    container_name: vsftpd
    environment:
      FTP_USER: ftpuser
      FTP_PASS: ftppass
    ports:
      - "20:20"
      - "21:21"
      - "21100-21110:21100-21110"

@adamdecaf
Copy link
Member

adamdecaf commented Mar 13, 2025 •
edited
Loading

That image looks to have some problems when new directories are created. The image isn't creating them with 0777 (even with LOCAL_MASK of 022) so the operations are failing. I do see them created on the filesystem though, but the server is responding with permission errors.

Claude response 
Looking at your directory permissions, I can see the root issue clearly now. The `/home/vsftpd/ftpuser/2025` directory is owned by `root:root` with `700` permissions (`drwx------`), which means only root can access it.

For your FTP users to create directories and upload files, you need to make these simple changes:

1. **First, fix the existing permissions**:
   # Change ownership of the ftpuser directory and all subdirectories to ftp:ftp
   chown -R ftp:ftp /home/vsftpd/ftpuser
   
   # Set proper permissions (read, write, execute for owner, and at least read+execute for others)
   chmod -R 755 /home/vsftpd/ftpuser

2. **For a permanent fix to your Docker image**, modify the `run-vsftpd.sh` script:
   # Find this line:
   chown -R ftp:ftp /home/vsftpd/
   
   # Make sure it runs AFTER the user directories are created, and ensure it's recursive
   # The current issue might be that directories are created after the chown command runs

3. **Update the umask value** in your container environment:
   LOCAL_UMASK=022
   This will ensure newly created directories have 755 permissions and files have 644 permissions.

These three changes should allow your FTP users to create directories and upload files without any issues, while maintaining the general structure of the original container.

I see the issue now. The script creates and sets ownership correctly, but something is changing the ownership after that step runs.

Let's investigate what's likely happening:

1. The script creates `/home/vsftpd/${FTP_USER}` with proper ownership
2. But when an FTP client creates a subdirectory (like "2025"), it's being created with root ownership

This could be due to:

1. **Virtual users not mapping correctly to the ftp system user**
2. **Missing the guest_username configuration**

Here's a minimalist fix:

1. **Add guest_username setting** to vsftpd.conf:
   echo "guest_username=ftp" >> /etc/vsftpd/vsftpd.conf

2. **Ensure local_umask is set properly**:
   echo "local_umask=022" >> /etc/vsftpd/vsftpd.conf

3. **Add a "fused" configuration**:
   echo "user_config_dir=/etc/vsftpd/users" >> /etc/vsftpd/vsftpd.conf
   mkdir -p /etc/vsftpd/users

   Then create a user-specific config file:
   echo "local_root=/home/vsftpd/${FTP_USER}" > /etc/vsftpd/users/${FTP_USER}
   echo "allow_writeable_chroot=YES" >> /etc/vsftpd/users/${FTP_USER}
   echo "write_enable=YES" >> /etc/vsftpd/users/${FTP_USER}

These configurations should ensure that:
1. Virtual users map correctly to the system ftp user
2. New directories and files have appropriate permissions
3. User-specific settings are applied correctly

This approach doesn't require changing the structure of the original container and focuses solely on the configuration issues.

There looks to be some improvements we could make in the docker image, but that image doesn't even build on my machine.

$ docker compose build
WARN[0000] /Users/adam/code/src/github.com/fauria/docker-vsftpd/docker-compose.yml: the attribute `version` is obsolete, it will be ignored, please remove it to avoid potential confusion 
[+] Building 3.5s (6/15)                                                                                                            docker:orbstack
 => [vsftpd internal] load build definition from Dockerfile                                                                                    0.1s
 => => transferring dockerfile: 1.17kB                                                                                                         0.0s
 => [vsftpd internal] load metadata for docker.io/library/centos:7                                                                             1.4s
 => [vsftpd internal] load .dockerignore                                                                                                       0.1s
 => => transferring context: 2B                                                                                                                0.0s
 => [vsftpd  1/11] FROM docker.io/library/centos:7@sha256:be65f488b7764ad3638f236b7b515b3678369a5124c47b8d32916d6487418ea4                     0.3s
 => => resolve docker.io/library/centos:7@sha256:be65f488b7764ad3638f236b7b515b3678369a5124c47b8d32916d6487418ea4                              0.1s
 => => sha256:be65f488b7764ad3638f236b7b515b3678369a5124c47b8d32916d6487418ea4 1.20kB / 1.20kB                                                 0.0s
 => => sha256:dead07b4d8ed7e29e98de0f4504d87e8880d4347859d839686a31da35a3b532f 529B / 529B                                                     0.0s
 => => sha256:eeb6ee3f44bd0b5103bb561b4c16bcb82328cfe5809ab675bb17ab3a16c517c9 2.75kB / 2.75kB                                                 0.0s
 => [vsftpd internal] load build context                                                                                                       0.2s
 => => transferring context: 4.10kB                                                                                                            0.0s
 => ERROR [vsftpd  2/11] RUN yum -y update && yum clean all                                                                                    1.4s
------                                                                                                                                              
 > [vsftpd  2/11] RUN yum -y update && yum clean all:                                                                                               
0.557 Loaded plugins: fastestmirror, ovl                                                                                                            
0.703 Determining fastest mirrors                                                                                                                   
0.831 Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os&infra=container error was                           
0.831 14: curl#6 - "Could not resolve host: mirrorlist.centos.org; Unknown error"

@adamdecaf
Copy link
Member

This server will be good to run some tests with, more servers covered gives us better confidence with the library.

#26 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@adamdecaf adamdecaf

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: add CreateUploadDirectories to mkdir inside UploadFile moov-io/go-ftp
2 participants
@adamdecaf @timchurchard