[Linux learning] – file splitting operation + file flattening operation

This article mainly introduces the knowledge points about linux, ubuntu, visual studio, and c++. Friends who are interested in [Linux learning] – file splitting operation + file flattening operation and linux splitting text files can read it from [ I want to eat the hardships of studying in graduate school] Contributed technical articles, I hope this technology and experience can help you solve the related technical problems you encounter.

Linux split text file

Table of Contents

One: Introduction

Two: Learning file splitting and file merging operations

3: File splitting + file merging (code implementation)

One: Introduction

Upload and download files in the network (file transfer function in the network), you need to use the operation of learning to split and merge files

2: Learn to split and merge files

The requirements are as follows

1. The program runs given a file path

2. Write a split file function, create a folder, split the given path into fragmented files and save them in this folder

3. The total size of the split N fragmented files is required to be the same as the original file

4. Write a flat file function to combine the previously split fragmented files into a complete file

Example:

A 1.jpg file, split until N x.temp fragment files inside a folder, and finally these fragment files can be combined into a complete final.jpg, and this picture file can still be double-clicked to view

Split: read reads one file, write writes multiple files

Splitting: read reads multiple files, write writes one file

Main issue:

1. Storage byte design

//1000 bytes 1KB split out is not 1024 but 1000, written according to hard disk manufacturer specifications
 //Manufacturers make hard drives according to 1000 1024 = 1.03KB
 char buf[1000] = { 0 };

2. The sorting problem of split and flattened file names

a. Linked list sorting (pointer pointing), the following sorting will appear

b. Need to be sorted into 1 2 3 4 5 order

I originally planned to use the built-in sorting, but there will be problems (sorting according to the string ASCII)

Because

 //Save all paths
     //list memory space utilization rate is high, suitable for frequent insertion and deletion
 list
   
     paths;//list container is used to save multiple paths (read multiple files, all files have paths and need to be saved) list
    
     ::iterator iter;//traverse iterator
    
   

is a string string, the string sorting is ASCII sorting, the following sorting will appear

So you can’t use the built-in sorting, you can ctrl+shift+/ comment multiple lines (shortcut key for multi-line comments in VS2019)

 //paths. sort();

 //Quick sort sort(start position, end position, sorting algorithm function pointer),

 /*
 * bool true false
 * return 0 exchange position
 * return 1 remains unchanged
 *
 bool cmp(string &a, string &b)
 {
 return a>b;
 }

 sort(paths. begin(), paths. end(), cmp)

 The big mouth faces forward and the big comes first (from big to small), the small mouth faces forward and the small comes first (from small to big)

 */

 /*cout <<"--------------After sorting-------------------------  -------" <<endl;

 for (iter = paths.begin(); iter != paths.end(); iter++)
 {
 cout <<*iter <<endl;
 }*/

c. The goal is to sort into 1 2 3 4 5 order, and finally use the following sorting to solve the problem

//refactor list
   
     Sorting of the path list sort paths.sort( [](const string& a, const string& b) { //String sorting -> integer sorting find to get the subscript first convert to char type and then atoi to integer return atoi(a.substr  (10, a.find(".")).c_str()) 

Three: file split + file combination (code implementation) 

The goal is to split the 3.jpg image file and put the files together into the 123 folder

Code implementation:

#include
   
     #include
    
      #include
     
       #include
      
        #include
       
         #include
        
          #include
         
           #include
          
            #include
           
             #include
           
              using namespace std; // Split files: read one file, write (create) multiple files int SplitFile(char filepath[]) { int readfd = 0; int writefd = 0; // return value int resbuf = 0; /  /1000 bytes of 1KB are not 1024 but 1000, which is written according to the specifications of the hard disk manufacturer//The hard disk produced by the manufacturer is made according to 1000 1024 = 1.03KB char buf[1000] = { 0 }; char writePath[  30] = { 0 }; //The first file is set to 1 int num = 1; umask(0); readfd = open(filepath, O_RDONLY, 0777); if (readfd  0) { //First file: 1.temp sprintf(writePath, "/root/123/%d.  temp", num); cout <<"writePath = " <<writePath < 0) { // split the file, close the write first, the file has not been read close(writefd); // write one file at a time, num++  One is 2.temp file num++; //Clear the read content bzero(buf, sizeof(buf)); //Clear the path, because a new path will be generated next time num++ bzero(writePath, sizeof(writePath)); }  } //File splitting first closes writing and then closes reading close(readfd); return num; } return 0; } //Combine files: read multiple files, write one file int CohereFile(char filepath[]) { int readfd =  0; int writefd = 0; int resbuf = 0; //A buf holds a piecemeal temp file char buf[1000] = { 0 }; //Read path char writePath[30] = { 0 }; //Write path  char readPath[30] = { 0 }; //Save all paths//list memory space utilization is high, suitable for frequent insertion and deletion of lists
             
               paths;//list container is used to save multiple paths (read multiple files, all files have paths and need to be saved) list
              
               ::iterator iter;//traversal iterator umask(0); //structure pointer DIR DIR* dir = NULL; //structure pointer dirent struct dirent* pdir = NULL; if ((dir = opendir(filepath))  == NULL) { perror("opendir error"); } else { while ((pdir = readdir(dir)) != NULL) { // /root/123/1.temp /root/123/2.temp/  root/123/3.temp if (strcmp(pdir->d_name, ".") == 0 || strcmp(pdir->d_name, "..") == 0) { continue; } bzero(readPath, sizeof  (writePath)); //Complete path splicing strcat(readPath, filepath); strcat(readPath, "/"); strcat(readPath, pdir->d_name); //The path of each .temp file is stored in the linked list paths.push_back  (readPath); } cout <<"paths.size() = " <<paths.size() <<endl; //test print path//for (iter = paths.begin(); iter != paths.end  (); iter++) //{ // cout <<*iter <<endl; //} // For example, the path of /root/123/1.temp /root/123/10.temp only needs to get 1 10 //Refactor  list
               
                 Sorting of the path list sort paths.sort( [](const string& a, const string& b) { //String sorting -> integer sorting find to get the subscript first convert to char type and then atoi to integer return atoi(a.substr  (10, a.find(".")).c_str()) 0) { cout <<"Split file completed" <<endl; cout <<"Start to join file" < 0) { cout <<"Flatten file completed" <<endl; } else { cout <<"Flatten file failed" <<endl; } } else { cout <<  "Failed to split file" <<endl; } return 0; }
               
              
             
            
           
          
         
        
       
      
     
    
   

View results:

Open the 123 folder and find that the files have been split successfully and the files have been merged successfully

You can double-click to view the files that are stitched together at the same time (there is no problem with splitting and stitching files)

The copyright of this article “[Linux Learning] – File Splitting Operation + File Combining Operation” belongs to the pain of postgraduate study, citing [Linux Learning] – File Splitting Operation + File Merging Operation must follow CC 4.0 BY-SA Copyright Agreement.