I noticed on my upload script that when a file is too big => ~ 10 MB this creates the following error:
Local directory \\customer_data\001952\info\ contains 1 file(s) to be uploaded.
Exception calling "Check" with "0" argument(s): "Lost connection.
Timeout detected. (data connection)
Could not retrieve directory listing
Copying files to remote side failed."
At \customer_data\EdiCustomerDataUp.ps1:66 char:17
+ $transferResult.Check()
+ ~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : SessionRemoteException
Hence the scripts stops and doesn't go further.
I must relaunch the script and remove temporarily from my variable loop the problematic folder in order to work it correctly. I however set the timeout limit to 5 Minutes but this doesn't change anything:
# Open the FTP Session
$session = New-Object WinSCP.Session
$session.Open($sessionOptions)
$session.Timeout = New-TimeSpan -Minutes 5
# Upload the files
$transferOptions = New-Object WinSCP.TransferOptions
$transferOptions.TransferMode = [WinSCP.TransferMode]::Automatic
$transferOptions.FileMask = "*.txt, *.csv, *.xlsx";
It is always the same .txt file that trigger the problem on my local folder:
Here is my session log:
< 2021-11-22 20:20:22.423 Script: Connected
. 2021-11-22 20:20:22.423 Connected
. 2021-11-22 20:20:22.423 --------------------------------------------------------------------------
. 2021-11-22 20:20:22.423 Using FTP protocol.
. 2021-11-22 20:20:22.423 Doing startup conversation with host.
< 2021-11-22 20:20:22.423 Script: Starting the session...
> 2021-11-22 20:20:22.423 PWD
< 2021-11-22 20:20:22.439 257 "/" is current directory.
. 2021-11-22 20:20:22.439 Getting current directory name.
. 2021-11-22 20:20:22.439 Startup conversation with host finished.
< 2021-11-22 20:20:22.439 Script: Session started.
< 2021-11-22 20:20:22.439 Script: Active session: [1] ************
> 2021-11-22 20:20:22.595 Script: pwd
< 2021-11-22 20:20:22.595 Script: /
> 2021-11-22 20:20:22.673 Script: put -delete -nopermissions -preservetime -transfer="automatic" -filemask="*.txt, *.csv, *.xlsx" -- "\\customer_data\001952\info\*" "/001952/info/"
. 2021-11-22 20:20:22.673 Copying 1 files/directories to remote directory "/001952/info/" - total size: 0
. 2021-11-22 20:20:22.673 PrTime: Yes; PrRO: No; Rght: rw-r--r--; PrR: No (No); FnCs: N; RIC: 0100; Resume: S (102400); CalcS: No; Mask:
. 2021-11-22 20:20:22.673 TM: M; ClAr: No; RemEOF: No; RemBOM: No; CPS: 0; NewerOnly: No; EncryptNewFiles: Yes; ExcludeHiddenFiles: No; ExcludeEmptyDirectories: No; InclM: *.txt, *.csv, *.xlsx; ResumeL: 0
. 2021-11-22 20:20:22.673 AscM: *.*html; *.htm; *.txt; *.php; *.php3; *.cgi; *.c; *.cpp; *.h; *.pas; *.bas; *.tex; *.pl; *.js; .htaccess; *.xtml; *.css; *.cfg; *.ini; *.sh; *.xml
. 2021-11-22 20:20:22.673 Retrieving listing to detect timezone difference
. 2021-11-22 20:20:22.673 Retrieving directory listing...
> 2021-11-22 20:20:22.673 CWD /001952/info
< 2021-11-22 20:20:22.705 250 CWD command successful.
> 2021-11-22 20:20:22.705 PWD
< 2021-11-22 20:20:22.720 257 "/001952/info" is current directory.
> 2021-11-22 20:20:22.720 TYPE A
< 2021-11-22 20:20:22.736 200 Type set to A.
> 2021-11-22 20:20:22.736 PASV
< 2021-11-22 20:20:22.767 227 Entering Passive Mode (194,25,152,107,216,222).
> 2021-11-22 20:20:22.767 LIST
. 2021-11-22 20:20:22.767 Connecting to 194.25.152.107:55518 ...
. 2021-11-22 20:20:37.374 Timeout detected. (data connection)
. 2021-11-22 20:20:37.374 Could not retrieve directory listing
< 2021-11-22 20:20:37.374 Script: Lost connection.
< 2021-11-22 20:20:37.374 Timeout detected. (data connection)
< 2021-11-22 20:20:37.374 Could not retrieve directory listing
< 2021-11-22 20:20:37.374 Copying files to remote side failed.
This is actually how the log looks when it's successful.
I can retrieve the FTP remote directory listing and I can see the content in WinSCP GUI (it takes some times as the folder has a huge number of files):
2021-11-23 08:41:02.247 Using FTP protocol.
. 2021-11-23 08:41:02.262 Doing startup conversation with host.
< 2021-11-23 08:41:02.262 Script: Starting the session...
> 2021-11-23 08:41:02.262 PWD
< 2021-11-23 08:41:02.262 257 "/" is current directory.
. 2021-11-23 08:41:02.262 Getting current directory name.
. 2021-11-23 08:41:02.262 Startup conversation with host finished.
< 2021-11-23 08:41:02.262 Script: Session started.
< 2021-11-23 08:41:02.262 Script: Active session: [1] ***********
> 2021-11-23 08:41:02.340 Script: pwd
< 2021-11-23 08:41:02.340 Script: /
> 2021-11-23 08:41:02.418 Script: put -delete -nopermissions -preservetime -transfer="automatic" -filemask="*.txt, *.csv, *.xlsx" -- "\\wipfs02\data\edi\live\wip\customer_data\001952\info\*" "/001952/info/"
. 2021-11-23 08:41:02.418 Copying 3 files/directories to remote directory "/001952/info/" - total size: 0
. 2021-11-23 08:41:02.418 PrTime: Yes; PrRO: No; Rght: rw-r--r--; PrR: No (No); FnCs: N; RIC: 0100; Resume: S (102400); CalcS: No; Mask:
. 2021-11-23 08:41:02.418 TM: M; ClAr: No; RemEOF: No; RemBOM: No; CPS: 0; NewerOnly: No; EncryptNewFiles: Yes; ExcludeHiddenFiles: No; ExcludeEmptyDirectories: No; InclM: *.txt, *.csv, *.xlsx; ResumeL: 0
. 2021-11-23 08:41:02.418 AscM: *.*html; *.htm; *.txt; *.php; *.php3; *.cgi; *.c; *.cpp; *.h; *.pas; *.bas; *.tex; *.pl; *.js; .htaccess; *.xtml; *.css; *.cfg; *.ini; *.sh; *.xml
. 2021-11-23 08:41:02.418 Retrieving listing to detect timezone difference
. 2021-11-23 08:41:02.418 Retrieving directory listing...
> 2021-11-23 08:41:02.418 CWD /001952/info
< 2021-11-23 08:41:02.434 250 CWD command successful.
> 2021-11-23 08:41:02.434 PWD
< 2021-11-23 08:41:02.434 257 "/001952/info" is current directory.
> 2021-11-23 08:41:02.434 TYPE A
< 2021-11-23 08:41:02.450 200 Type set to A.
> 2021-11-23 08:41:02.450 PASV
< 2021-11-23 08:41:02.450 227 Entering Passive Mode (194,25,152,107,217,42).
> 2021-11-23 08:41:02.450 LIST
. 2021-11-23 08:41:02.450 Connecting to 194.25.152.107:55594 ...
< 2021-11-23 08:41:02.497 125 Data connection already open; Transfer starting.
. 2021-11-23 08:45:56.340 -rwxrwxrwx 1 owner group 350 Nov 11 8:07
. 2021-11-23 08:45:56.340 -rwxrwxrwx 1 owner group 450 Nov 18 8:12
< 2021-11-23 08:45:56.340 226 Transfer complete.
. 2021-11-23 08:45:56.340 Data connection closed
. 2021-11-23 08:45:56.340 Directory listing successful
. 2021-11-23 08:45:56.355 Detecting timezone difference...
. 2021-11-23 08:45:56.355 Retrieving file information...
> 2021-11-23 08:45:56.355 PWD
< 2021-11-23 08:45:56.355 257 "/006839/info" is current directory.
> 2021-11-23 08:45:56.355 CWD /006839/info/availability_006839.txt
< 2021-11-23 08:45:56.371 550
> 2021-11-23 08:45:56.371 TYPE I
< 2021-11-23 08:45:56.371 200 Type set to I.
> 2021-11-23 08:45:56.371 SIZE /006839/info/availability_006839.txt
< 2021-11-23 08:45:56.387 213 641473
> 2021-11-23 08:45:56.387 MDTM /006839/info/availability_006839.txt
< 2021-11-23 08:45:56.387 213 20211121231105
. 2021-11-23 08:45:56.387 Retrieving file information successful
. 2021-11-23 08:45:56.387 No timezone difference detected using file /006839/info/availability_006839.txt (Listing: 2021-11-21T23:11:00.000Z, UTC: 2021-11-21T23:11:00.000Z)
. 2021-11-23 08:45:56.387 File: '\\wipfs02\data\edi\live\wip\customer_data\006839\info\availability_006839.txt' [2021-11-22T20:08:33.827Z] [641472]
. 2021-11-23 08:45:56.418 Copying "\\wipfs02\data\edi\live\wip\customer_data\006839\info\availability_006839.txt" to remote directory started.
. 2021-11-23 08:45:56.418 Ascii transfer mode selected.
. 2021-11-23 08:45:56.418 Starting upload of \\wipfs02\data\edi\live\wip\customer_data\006839\info\availability_006839.txt
> 2021-11-23 08:45:56.418 TYPE A
< 2021-11-23 08:45:56.434 200 Type set to A.
> 2021-11-23 08:45:56.434 PASV
< 2021-11-23 08:45:56.434 227 Entering Passive Mode (194,25,152,107,217,13).
> 2021-11-23 08:45:56.434 STOR availability_006839.txt
. 2021-11-23 08:45:56.434 Connecting to 194.25.152.107:55565 ...
< 2021-11-23 08:45:56.481 125 Data connection already open; Transfer starting.
< 2021-11-23 08:45:56.544 226 Transfer complete.
> 2021-11-23 08:45:56.544 MDTM 20211122200833 availability_006839.txt
< 2021-11-23 08:45:56.544 213 20211122200833
. 2021-11-23 08:45:56.560 Upload successful
. 2021-11-23 08:45:56.560 Transfer done: '\\wipfs02\data\edi\live\wip\customer_data\006839\info\availability_006839.txt' => '/006839/info/availability_006839.txt' [641472]
. 2021-11-23 08:45:56.560 Deleting successfully uploaded source file "\\wipfs02\data\edi\live\wip\customer_data\006839\info\availability_006839.txt".
. 2021-11-23 08:45:56.560 File: '\\wipfs02\data\edi\live\wip\customer_data\006839\info\FW21_SEASON_Items_CarharttWIP_EAN.csv' [2021-11-23T05:01:35.301Z] [2237102]
. 2021-11-23 08:45:56.576 Copying "\\wipfs02\data\edi\live\wip\customer_data\006839\info\FW21_SEASON_Items_CarharttWIP_EAN.csv" to remote directory started.
. 2021-11-23 08:45:56.576 Binary transfer mode selected.
. 2021-11-23 08:45:56.576 Starting upload of \\wipfs02\data\edi\live\wip\customer_data\006839\info\FW21_SEASON_Items_CarharttWIP_EAN.csv
> 2021-11-23 08:45:56.576 TYPE I
< 2021-11-23 08:45:56.591 200 Type set to I.
> 2021-11-23 08:45:56.591 PASV
< 2021-11-23 08:45:56.607 227 Entering Passive Mode (194,25,152,107,217,14).
> 2021-11-23 08:45:56.607 STOR FW21_SEASON_Items_CarharttWIP_EAN.csv
. 2021-11-23 08:45:56.607 Connecting to 194.25.152.107:55566 ...
< 2021-11-23 08:45:56.607 150 Opening BINARY mode data connection.
< 2021-11-23 08:45:56.810 226 Transfer complete.
> 2021-11-23 08:45:56.810 MDTM 20211123050135 FW21_SEASON_Items_CarharttWIP_EAN.csv
< 2021-11-23 08:45:56.826 213 20211123050135
. 2021-11-23 08:45:56.826 Upload successful
. 2021-11-23 08:45:56.826 Transfer done: '\\wipfs02\data\edi\live\wip\customer_data\006839\info\FW21_SEASON_Items_CarharttWIP_EAN.csv' => '/006839/info/FW21_SEASON_Items_CarharttWIP_EAN.csv' [2237102]
. 2021-11-23 08:45:56.826 Deleting successfully uploaded source file "\\wipfs02\data\edi\live\wip\customer_data\006839\info\FW21_SEASON_Items_CarharttWIP_EAN.csv".
. 2021-11-23 08:45:56.841 File: '\\wipfs02\data\edi\live\wip\customer_data\006839\info\FW22_INFO_Items_CarharttWIP_EAN.csv' [2021-11-23T05:16:05.992Z] [1509062]
. 2021-11-23 08:45:56.841 Copying "\\wipfs02\data\edi\live\wip\customer_data\006839\info\FW22_INFO_Items_CarharttWIP_EAN.csv" to remote directory started.
. 2021-11-23 08:45:56.841 Binary transfer mode selected.
. 2021-11-23 08:45:56.841 Starting upload of \\wipfs02\data\edi\live\wip\customer_data\006839\info\FW22_INFO_Items_CarharttWIP_EAN.csv
> 2021-11-23 08:45:56.857 TYPE I
< 2021-11-23 08:45:56.857 200 Type set to I.
> 2021-11-23 08:45:56.857 PASV
< 2021-11-23 08:45:56.872 227 Entering Passive Mode (194,25,152,107,217,15).
> 2021-11-23 08:45:56.872 STOR FW22_INFO_Items_CarharttWIP_EAN.csv
. 2021-11-23 08:45:56.872 Connecting to 194.25.152.107:55567 ...
< 2021-11-23 08:45:56.872 150 Opening BINARY mode data connection.
< 2021-11-23 08:45:57.044 226 Transfer complete.
> 2021-11-23 08:45:57.044 MDTM 20211123051605 FW22_INFO_Items_CarharttWIP_EAN.csv
< 2021-11-23 08:45:57.060 213 20211123051605
. 2021-11-23 08:45:57.076 Upload successful
. 2021-11-23 08:45:57.076 Transfer done: '\\wipfs02\data\edi\live\wip\customer_data\006839\info\FW22_INFO_Items_CarharttWIP_EAN.csv' => '/006839/info/FW22_INFO_Items_CarharttWIP_EAN.csv' [1509062]
. 2021-11-23 08:45:57.076 Deleting successfully uploaded source file "\\wipfs02\data\edi\live\wip\customer_data\006839\info\FW22_INFO_Items_CarharttWIP_EAN.csv".
. 2021-11-23 08:45:57.076 File: '\\wipfs02\data\edi\live\wip\customer_data\006839\info\WIP_Sales_Shipment_Export_21112371322.txt' [2021-11-23T06:13:22.337Z] [558]
. 2021-11-23 08:45:57.091 Copying "\\wipfs02\data\edi\live\wip\customer_data\006839\info\WIP_Sales_Shipment_Export_21112371322.txt" to remote directory started.
. 2021-11-23 08:45:57.091 Ascii transfer mode selected.
. 2021-11-23 08:45:57.091 Starting upload of \\wipfs02\data\edi\live\wip\customer_data\006839\info\WIP_Sales_Shipment_Export_21112371322.txt
> 2021-11-23 08:45:57.091 TYPE A
< 2021-11-23 08:45:57.107 200 Type set to A.
> 2021-11-23 08:45:57.107 PASV
< 2021-11-23 08:45:57.107 227 Entering Passive Mode (194,25,152,107,217,16).
> 2021-11-23 08:45:57.107 STOR WIP_Sales_Shipment_Export_21112371322.txt
. 2021-11-23 08:45:57.107 Connecting to 194.25.152.107:55568 ...
< 2021-11-23 08:45:57.122 150 Opening ASCII mode data connection.
< 2021-11-23 08:45:57.122 226 Transfer complete.
> 2021-11-23 08:45:57.122 MDTM 20211123061322 WIP_Sales_Shipment_Export_21112371322.txt
< 2021-11-23 08:45:57.138 213 20211123061322
. 2021-11-23 08:45:57.169 Upload successful
. 2021-11-23 08:45:57.169 Transfer done: '\\wipfs02\data\edi\live\wip\customer_data\006839\info\WIP_Sales_Shipment_Export_21112371322.txt' => '/006839/info/WIP_Sales_Shipment_Export_21112371322.txt' [558]
. 2021-11-23 08:45:57.169 Deleting successfully uploaded source file "\\wipfs02\data\edi\live\wip\customer_data\006839\info\WIP_Sales_Shipment_Export_21112371322.txt".
. 2021-11-23 08:45:57.169 Copying finished: Transferred: 4.388.194, Elapsed: 0:00:05, CPS: 6.354.375/s
> 2021-11-23 08:45:57.279 Script: exit
. 2021-11-23 08:45:57.279 Script: Exit code: 0
. 2021-11-23 08:45:57.279 Disconnected from server
Could you help me to avoid this error? Thanks!
