File Processing

The file handling won't be much different than that from PCBoard. However I need to research the PPEs for a final decision :). Internally it'll be different. PCBoard works basically just with a text file. IcyBoard will have a more database like approach.

I want to provide a file scanner/repacker to work with file bases. It should support the most common formats out of the box. I want to provide as much code in Rust as possible which is a problem in that area.

That's why I've begun working on compression algorithms. I ported zip compression methods Shrink, Reduce and Implode to zip-rs (excellent blog post about these methods are here: https://www.hanshq.net/zip2.html).

I've implemented an ARJ decompressor and started working on ARC. There seems to be a working RAR library that I can just use as well as LZH/LHA. So it'll support 7Z, ZIP, ARJ, RAR, LHA/Z and ARC out of the box. The rest needs to be done with external packers. If you think that I need to support another compression format let me know :).

Providing these out of the box will make it easier to set up icy board on various operating systems. Setting up pcboard is very complex - I want to make at least some things easier.

What I basically plan is:

user uploads -> search for file_id.diz -> user continues (or needs to enter a description)

This is the basic case. But optionally, the uploaded files should be processed further. The upload processor should be configurable and should be able to do the following tasks in background:

• Decompress the uploaded file to a temporary directory
• Deeper file analyzation (like ENIGMA BBS)
  
• Remove BBStros & ad files
  
• Remove BBS ads from FILE_ID.DIZ/ANS etc.
• Custom steps (add own bbstros, running a virus scan etc.)
  
• Recompress the file

Each step should be configurable - for example, removing BBS ads doesn't make sense in a "bbstro" file area :). At least decompressing/recompression makes much sense. By converting to the .7z format the bbs files take up ~25% less space. And not every user is able to uncrunch let's say 'ZOO' files.

I can need help in that area: Which tools are there that do something like this?

Esp. how to remove the BBStros? I have some archives of BBStros where I just can take the checksum + filename - but I'm sure that there are some pre existing approaches that are more sophisticated. There are some files that are "generated" so not everything would be catched with a checksum approach.

Does it make sense to invest some time in the virus scan area? I speak about DOS viruses here - they're basically finished :). Getting an up2date virus database from f-prot or something should be possible. Are there any modern virus scanners that search for DOS viruses?