Large Part Files

[Jerome]

Does anybody know whether the entire part file is read into memory or is it read from the hard drive as needed?

I had a Comm error last week while doing a carving and it was a 678,190 line file. It got me to wondering if there was a blip reading the file.

Thanks
Jerome

[adrianm]

There is information on it in the ProgHand.pdf file that is installed with the SB3 software.



When a Part File is read by your ShopBot Software, the program's first priority is to process all the tool movement commands in such a way that they can be most efficiently and smoothly executed. To make execution smooth, as the software reads through the file it accumulates tool movement commands and stores them into memory in lists in an efficient pre-processed form. We call this stored list of moves a 'movement stack'. At the time the movement stack is being stored, the moves are also analyzed to determine where acceleration and deceleration ramps should be placed, and how other features such as 'tabbing' should be applied. Establishing the movement stack is always the first priority of the software as files are read.
In most cases, arranging your file into movement stacks is completely handled by ShopBot, so you won't need to think about it. However, in certain situations it can be helpful to take control of how movement blocks are handled so we'll explain this process here.
As a Part File is being read, when the first tool move is encountered (e.g. an M2) a movement block is created. The software will then continue to read lines accumulating movement instructions. No movement will be executed until the whole stack has been read.
The end of the stack is determined by:
1. Reaching an Command that is not a Move, Cut, or SwitchON Command, or a special case Insert.
2. Or, by reaching the end of stack memory.
After the stack is read, all the moves in it will execute. If the block is terminated by a non-Move Command, then that next command will be executed. Following it, when a new Move Command is encountered in the file, the next movement block will be acquired and executed (e.g. a J2, is a Jog and not a Move; the Jog ends the present stack and the stack is executed, then the Jog is executed as a single stack, point-to-point move).
If reaching the limit of available memory punctuated the reading of the Part File, then at the end of execution of the stack, the bit will be pulled to the 'Safe-Z' height (if this feature is active). After the pull-up, an additional stack will be read and then the bit will be re-inserted for cutting that stack of moves.
There are a couple of situation where you may wish to interrupt the automatic handling of movement blocks. The [SC, 2] Command can be used to explicitly put a stack-end and stack-restart [SC,2] at a specific location in a Part File.
The reason you may want to consider controlling stacks is because it may allow you to prevent an interruption in cutting to load more of the file from occurring at an awkward location. By placing SC,2's at locations in a file before memory is used up by the current block, the block will be executed and a new one started. You can thus force the blocking to occur at the location in the file where it will not have an adverse effect on the cutting.

[Jerome]

Adrian

Thanks for the response.

I read it over several times and still can't tell whether this happens in memory or is reading from the hard drive during the processing.

Could be my age is catching up with me at 68 and counting.

Jerome

[steve_g]

Jerome

All I can tell you is that occasionally my bot will go to the safe Z, pause and load additional code. I think this normally happens quite transparently but due to the fact that I run off a network drive, it isn't always very fast.

SG

[Jerome]

I too am running off a netwmork drive and I also notice a pause every so often when doing 3D carving.

This is what got me to wondering if the network caused my comm error.

If I decide that is the problem I will start saving my files on the local drive.

Jerome

[ssflyer]

I get the pause every once in awhile running a large 3D file from my local drive, too. It just seems to be buffering in more of the file. this may have to do with the amount of RAM installed...

[Bob Eustace]

Yes we too get that pausing on 3D files - heart stopping the first time it happens 5 hours or so into a job and you HAVEN'T followed Jack's good advice and written down the true start co-ordinates!

Could Brady possibly tell us if this still happens with the fast Asus he is testing please?

Another thought - has anyone used a SSD - solid state drive? These are built into Laguna now and might speed things up.

[steve_g]

"If I decide that is the problem I will start saving my files on the local drive."


The reason I started using a network drive was that I was using a thumb drive to transfer files from my design computer to my bot computer... Then I'd make some hand coded changes to the run file and or a design change on the office computer and end up with three versions of the same file, none of which were completely correct. I now save any changes to the one network location. An added benefit is that my 1 terabyte drive is mirrored... a backup system I don't have to think about!

SG

[adrianm]

The pause is just the next chunk of code being buffered into the stack memory. The more RAM you have the longer the pause because there is more being read in but also you are far less likely to see any pause at all unless it's a very large file.

My first controller PC only had 512mb of memory and that would pause for a short while quite often. Now I have 2gb in the latest PC and I only get a pause on the largest 3D files.

Anything that would speed up the transfer process from disk to memory (such as an SSD) should reduce the length of the pause I would think.

[bcondon]

I NEVER run off the memory stick because I always get comm errors.

In fact, when I am unloading my stick after executing remove device
icon, it is quite common that I get a comm error of which time, I shut down SB3 software and bring it back up.

I always design on one machine which is on the internet, load SBF files onto a memory stick and bring only those files on my shopbot machine. I have noticed many times that there is some sort of conflict on the USB channels , get a comm error. SO instead of spending hours of detective work, I just change my process and merrily go on my way.

I also don't change my software with every upgrade, because that makes people spend days on end figuring why it does not work the same any more...
My SB3 software is 2006 version and it does a great job.... for me.

Thanks
Bob

[Bob Eustace]

I have noticed many times that there is some sort of conflict on the USB channels , get a comm error. SO instead of spending hours of detective work, I just change my process and merrily go on my way.

Thanks
Bob

Bob we are working the same way as you BUT often SB3 just wont shut down until we reboot the computer. I dont understand what you are doing when you say "I just change my process"?

Could you possibly outline what you are currently doing please?

Steve - is your network drive hooked up via an ethernet cable? Do you get comms problems?

[Brady Watson]

Could Brady possibly tell us if this still happens with the fast Asus he is testing please?


I have been running the same XP Pro computer (ASUS or Shuttle MoBo...I'm not 100% sure at the moment) without a hiccup for at least 5 years. It was pretty fast when I put it together (I build my own computers from specific components) - and my tool runs a Belkin 2.0 PCI to USB card with an El Cheapo Radio Shack self powered hub along with an IOGear USB to Serial converter. COM speed is around 87-90% these days...with 20' of USB cable (which I do NOT recommend any of you do!)

I've been so busy in the past year or so I haven't had the time to setup a network from the office to the shop, so I am 'sneaker networking' at the moment with a USB drive. Since it is XP, I have to eject the drive every time or things can get flaky...and I always plug the drive into a different port (as in - not even on the Belkin card let alone the hub) just in case something goofy happens. I eject it and physically take it out before I machine.

I'm proud to say that a few weeks ago, my 2001 PRT with 7.2 Alpha drive & 5hp Colombo did 10 MILES of cutting in 4 days...and I got about 8hrs sleep total over those 4 days...Typical rush job. I had to take 4 front end loads of maple chips out of the shop...It was no joke. Pics later...I am sitting in an airport trying to get home early to beat out the storm heading for my crib :D

-B

[steve_g]

Steve - is your network drive hooked up via an ethernet cable? Do you get comms problems?


Bob

My Network Attached Storage (NAS) drive is attached to my network at the router with a cat 5 cable (Ethernet) ... all computers on my network can be mapped to see the device. My device has two bays for standard SATA drives and can be setup as one drive or as a mirrored drive (both drives contain the same information). My only complaint with the device is that it's not as fast as the other drives in my computers. My network is very stable and I seldom have issues with it... I also seldom (knock on wood) have com issues. I attribute my lack of issues to my total disregard to grounding of anything!(???).

http://techgage.com/article/d-link_dns-323_nas/ (http://techgage.com/article/d-link_dns-323_nas/) Here's an article if you want to know more about the network drives.

SG

[jerry_stanek]

My NAS is stable to but I copy to my desktop to run a file

[steve_g]

Jerry

I understand why you would want to copy it over, I don't because I was getting so many copies of various files I was not sure which was the latest and greatest! Do you delete the version you copied over after you run the job? So far the only disadvantage I can come up with is it takes a little longer to fill the queue.

SG

[Brady Watson]

I had to catch a plane back East before this FrankenBerry Storm or whatever they are calling it...so I wanted to just update - Yes, it is normal for the tool to stop and refill the command buffer during long 3D files or files that have a lot of lines. It doesn't matter how much RAM you have on the computer - RAM makes a difference on the max number of lines you can have in a file overall - but the buffer size of command lines should be the same regardless of RAM size.

Recently on a long/large (5x12') relief the buffer only refilled while running maybe once...but a during a 2D toolpath that was projected on the 3D relief the buffer refilled about 6 times - and the number of lines greatly exceeded that of the 3D raster toolpath. The buffer is refilled every (I think...don't quote me on this) 10,000 lines or so.

The command buffer will be cleared out and refilled during any 'interrupt event' - meaning you hit the S key, spacebar etc. This is because you may pause the file and then Insert a command such as MS or even modify VR settings in the middle of a file. Since there are look ahead calculations that have to be done to reflect the new changes/tweaks (so the software can anticipate moves before they need to happen), the buffer is cleared out and refilled. This occurs even if you just stop the tool and then hit resume.

-B

[jerry_stanek]

Yes I delete it once I run it but leave it on the NAS I used to run directly off the NAS but one time our internet went down and the router reset and the NAS stumbled.

[Jerome]

This subject seems to be a common problem.

I work in aspire on a Windows 7 computer. I save it to my network drive and load and run it from my xp computer which runs the shopbot.

The last comm error I got, the spindle had retracted so I think it was refilling the buffer when it crashed. This is what got me to thinking I had a blip on the network.

I ran a large carving file Saturday which took about 5 hours to run. Before beginning I copied the part file to the local hard drive and it ran with no errors.

My plan is to use the network drive for most part files but the long carving files will be copied to the local drive.

My thanks to everyone who responded to this thread.

Jerome

[Brady Watson]

The last comm error I got, the spindle had retracted so I think it was refilling the buffer when it crashed.

The Z doesn't usually go high when the machine pauses to fill up the buffer. It usually just stops right where it is in XYZ, then resumes, picking up where it left off.

-B

[steve_g]

Brady

I'm pretty sure my experience is that it retracts to the safe Z setting while filling the buffer.

SG

[Brady Watson]

I must be running a different version of SB3 than most :o

Thanks for the correction...

-B

[bcondon]

Bob E.

My process is to load my files before bringing up SB3 if possible.

If not, I simply
1. put in the finger stick
2. Load files with explorer
3. Disconnect drive - Here is where I would get the comm error.
4A if not comm error, just continue
4B If comm error, shut down the SB3 software (typically by hitting esc and then saying yes to shutdown)
5B. Restart SB3.. LOTS of time, it will not come up (and the reason is that SB3 did not shut down the process correctly)
6B. Control - Alt - Del will let you bring up the Task Manager
7B. FInd SB3 in the process list, select and then shut it down with the button.

You can not restart SB3 and I have not had any failures from thee on...

Thanks

Bob Condon

[garyc]

Bob...
The comm loss is fairly common with Dell (w/o PCI card) and other computers with less than robust USB controller chips. Since I am not able to convince most folks that their "expensive CNC robotic tool" could use a slightly better computer, I have them switch SB3 to preview mode when inserting and tranferring to/from a USB thumb drive. Set back to move/cut mode after removing drive.

Brady....
Yes you are!!!!! :D:D

[myxpykalix]

Bob C,
I'm a bit confused on your loading sequence.
2. Load files with explorer
3. Disconnect drive - Here is where I would get the comm error.

Does that mean you load them onto your shopbot control computer like on the desktop or someplace you can call them up? Or in some temporary memory?

I usually make my files in the house and carry them out on a thumbdrive and plug that into the computer and call up the file from within the shopbot program and run them directly from the drive. I've done it that way for years with no issues. :confused:

[garyc]

Jack....
Soooooo, you are the one spreading that bad rumor! :mad::mad::mad: Bob must have read the ShopBot Manual.

For everyone but Jack....
USB thumb drives tranfer rates are below that that SB3 comm rates require. To ensure proper operation, cutting files MUST be copied to a local drive on the control computer and run from there, NOT directly from the thumb (USB) drive.

[myxpykalix]

Really? Like the title of my other post "You can teach an old dog new tricks"!

I guess i did not realize that. I have never had a communication issue, maybe because i run things at a slower rate then the maximum transfer rate? What you are saying makes sense, i just never had any problems..:)

It might be like an old man driving a corvette at 30 mph:eek:

[Bob Eustace]

Bob & Gary

Modified my routine by always swapping to preview before removing/inserting. Interestingly did six signs in a row with no coms problems. Will stick with that for a while but will fiddle with Steves NAS idea. Gary do you think using an SSD would be better?

[steve_g]

Bob

FWIW, I'm going to alter my procedures and copy the file from my NAS drive to my bot control computer like Jerry S. does, and then delete it after running it. If I have made custom hand coded alterations to the file, I'll copy it back to the NAS drive before deleting. I believe this may be a better procedure that still solves my "multiple version" issues.

SG

[Bob Eustace]

Going great! Another six signs today and no hint of a comms failure. I'm stoked!