C# network downloader

C# network downloader

I have learned part of the basics of C# before, but I feel that I don’t know much about it. Many areas still need to be consolidated by doing a small demo, so here is this C# network downloader

Principle explanation

First of all, before we write code, we need to understand what is the principle of network downloading?
If you have studied the IO stream part in C#, or if you have a foundation in other languages, or if you have studied the basics of file IO in other languages, you must have understood that the data in our computers are all binary, so the data transmitted on the network is essentially the same.

When we learn file IO, we will all learn the read and write operations of files. The read operation (Output) reads the binary data in the file, and the write operation (Input) writes the binary data in the memory to the file on the hard disk.

Then the essence of network downloading is the reading and writing of files, and the steps are divided into the following steps:

1. Initiate a request to the server
2. The server receives the request and returns a response, and this response is a file stream data
3. The program receives the response and reads the binary data in the response body (file reading operation)
4. Write the read file binary data to the disk

code implementation

Now that you understand the principle, let’s start implementing the code!

namespace WebDownLoad
 {
     //A download task class
     public class DownLoadTask
     {

         public async Task Start(string url,string targetUrl)
         {
             try
             {
                 // 1. First create an HttpClient connection
                 // Since HttpClient implements the IDispose interface, we can recycle its resources
                 using HttpClient client = new HttpClient();
                 // 1.1. Some websites will be reverse crawled, so we need to set some parameters
                 client.DefaultRequestHeaders.Add("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safari/537.36 Edg/118.0.2088.61");

                 // 2. Make an asynchronous request
                 HttpResponseMessage response = await client.GetAsync(url);
                 // 3. Request successful
                 if (response.IsSuccessStatusCode)
                 {
                     // 3.1. Read file type
                     string? type = response.Content.Headers.ContentType?.MediaType;
                     // 3.2. Total size of files
                     long? totalSize = response.Content.Headers.ContentLength;

                     // 3.3. Get the file stream
                     //The size of the buffer area is 10 kb
                     // Current download size
                     long downloadSize = 0;
                     int length = 0;

                     using BufferedStream bufferedStream = new BufferedStream(
                         await response.Content.ReadAsStreamAsync());
                     //Buffer size 0.5KB
                     byte[] bufferSize = new byte[1024];
                     IEnumerable targetBuffer = new List();
                     string suffix = GetType(type);
                     while ((length = bufferedStream.Read(bufferSize, 0, bufferSize.Length)) != 0)
                     {
                         downloadSize += length;
                         long? progress = downloadSize * 100 / totalSize;
                         targetBuffer = targetBuffer.Concat(bufferSize.ToList());
                         await Console.Out.WriteAsync($"\rDownloading{progress}%");
                     }
                     await Console.Out.WriteLineAsync();
                     File.WriteAllBytes(targetUrl + Random.Shared.Next(10, 10000) + suffix, targetBuffer.ToArray());
                 }
                 // 4. Request failed
                 else
                 {
                     Console.WriteLine("Request to download failed");
                 }

             }catch (HttpRequestException e)
             {
                 Console.WriteLine($"Request to download failed: {e.Message}");
             }
         }

 //Detect the file type
         private string GetType(string type) {
             string suffix = "";
             if (type.Contains("jpeg"))
             {
                 suffix = ".jpg";
             }
             else if (type.Contains("application/octet-stream"))
             {
                 suffix = ".exe";
             }
             else if (type.Contains("png"))
             {
                 suffix = ".png";
             }
             else if (type.Contains("mp4"))
             {
                 suffix = ".mp4";
             }
             else if (type.Contains("avi"))
             {
                 suffix = ".avi";
             }
             else if (type.Contains("mp3"))
             {
                 suffix = ".mp3";
             }
             else if (type.Contains("mpeg"))
             {
                 suffix = ".m4a";
             }
             return suffix;
         }
     }
 }
 

The code I wrote above is actually a bit cumbersome. You can write it better without looking at my writing method. I am not familiar with many classes in C#, so the shortcomings of the overall code are still many.

In fact, in order to create a downloading effect here and allow console users to experience it, a lot of unnecessary operations have been done

In order to download multiple files at one time, asynchronous operations are used here to increase the download reception of the program. In fact, you can also use threads to implement it without asynchronous operations

One last experiment

using WebDownLoad;

 namespace WebDownLoad
 {
     public class Program
     {
         public async static Task Main(string[] args) {

             while(true)
             {
                 Console.WriteLine("Please enter the download address (if you enter 0 to exit):");
                 string url = Console.ReadLine();

                 if ("0".Equals(url))
                 {
                     break;
                 }
                 DownLoadTask task = new DownLoadTask();
                 task.Start(url, "E:\\Network Download\\");
             }
         }

     }
 }
 

There is nothing in the current folder

You can continue to download here, and there is another problem. We need to understand that in C#, asynchronous does not actually open a new thread. The underlying implementation of asynchronous in C# essentially uses switch goto to perform state jumps, that is, It will not actually speed up the processing, but it can increase the reception speed of the program, that is, the reception is very fast, but the processing remains unchanged. To speed up the processing, you still have to open a thread. Our thread itself is asynchronous, so this It may be better for the program to be implemented using threads