Redis, MemCached, MongoDB comparison
Recently, I have been studying server performance optimization and high-concurrency request access, investigating unstructured data (NoSQL) and memory acceleration (Cache), and redesigning the old platform services, striving to save costs by US$10,000 per month. The research projects mainly include Redis, MemCached, MongoDB, and Amazon’s DynamoDB Redis is an open source ANSI It is written in C language, supports network, can be memory-based and persistent log-type, Key-Value database, and provides APIs in multiple languages. The development work is currently hosted by VMware. 1) Data model As a Key-value database, Redis also provides a mapping relationship between keys and values. In addition to regular numerical values or strings, Redis key values can also be in one of the following forms: Lists Sets Sorted sets (ordered sets) Hashes (hash table) The data type of a key value determines the operations supported by the key value. Redis supports advanced atomic operations such as intersection, union, and difference of lists, sets, or ordered sets; at the same time, if the type of the key value is an ordinary number, Redis provides atomic operations such as increment. 2) Persistence: Redis usually stores data in memory or is configured to use virtual memory. Data persistence can be…
Mysql, sqlsever, oracle, redis, mongo, postgres and other database backup and recovery instructions organized_MySQL
The following only summarizes the simplest usage of each database backup and recovery. For more detailed parameters and configurations, please refer to the relevant information 1. mysql: Parameter description: $user $password $targethost $port $charset $dbname $table $filename Username Password ip(localhost) Port Encoding Database name Table name (only required for table-level backup) File name Backup: mysqldump –single-transaction -l -u $user -p'$password' -h $targethost -P $port –default-character-set=$charset $dbname $table > $filename Restore: mysql $dbname -u $user -p'$password' -h $targethost -P $port –default-character-set=$charset <$filename 2. sqlserver: Note: sqlserver2003 and below only support osql, sqlserver2005 and above support both osql and sqlcmd. The following methods are common to both sqlcmd and osql. Parameter description: $user $passwrod $instance $dbname $filename $diff_filename Username Password Instance name Database name Full backup file name Incremental backup file name Full backup: osql -U $user -P “$password” -d master -S $instance -Q “ALTER DATABASE $dbname SET OFFLINE WITH ROLLBACK IMMEDIATE; RESTORE DATABASE $dbname FROM DISK='$filename& #39; WITH REPLACE; ALTER DATABASE $dbname SET ONLINE” Incremental backup: osql -U $user -P “$password” -d master -S $instance -Q “ALTER DATABASE $dbname SET OFFLINE WITH ROLLBACK IMMEDIATE; RESTORE DATABASE $dbname FROM DISK='$ filename' WITH REPLACE,NORECOVERY; RESTORE DATABASE $dbname FROM DISK='$diff_filename' WITH RECOVERY; ALTER DATABASE…
How to connect Python to MySQL, MongoDB, Redis, memcache and other databases
I have been writing scripts in Python for some time, and I often operate databases (MySQL). Now I will sort out the operations on various databases. If there are new parameters later, I will add them in and gradually improve them. 1. Python operation MySQL: For details, see: [apt-get install python-mysqldb] The code is as follows: #! /bin/env python# -*- encoding: utf-8 -*-#————————– ————————————————– —# Purpose: example for python_to_mysql# Author: zhoujy# Created: 2013-06-14# update: 2013-06-14#—– ————————————————– ————————–import MySQLdbimport os #Establish a connection with the database system, format#cOnn= MySQLdb.connect(host=’localhost’,user=’root’,passwd=’123456′,db=’test’,port=3306, charset=’utf8′) #Specify the configuration file, determine the directory, or write the absolute pathcwd = os.path.realpath(os.path.dirname(__file__))db_cOnf= os.path.join(cwd, ‘ db.conf’)cOnn= MySQLdb.connect(read_default_file=db_conf,host=’localhost’,db=’test’,port=3306,charset=’utf8′) #SQL statement to be executedquery = ‘select id from t1’ #Get the operation cursorcursor = conn.cursor() #Execute SQLcursor.execute(query) #Get a record, each record is returned as a tuple, 3 is returned, and the cursor points to the second record. result1 = cursor.fetchone()for i in result1: print i#Return the number of affected rows print cursor.rowcount #Get the specified number of records, return each record as a tuple, return 1, 2, the cursor starts from the 2nd record, and the cursor points to the 4th record. result2 = cursor.fetchmany(2)for i in result2: for ii in…
Mysql, sqlsever, oracle, redis, mongo, postgres and other database backup and recovery instructions organized_MySQL
The following only summarizes the simplest usage of each database backup and recovery. For more detailed parameters and configurations, please refer to the relevant information 1. mysql: Parameter description: $user $password $targethost $port $charset $dbname $table $filename Username Password ip(localhost) Port Encoding Database name Table name (only required for table-level backup) File name Backup: mysqldump –single-transaction -l -u $user -p'$password' -h $targethost -P $port –default-character-set=$charset $dbname $table > $filename Restore: mysql $dbname -u $user -p'$password' -h $targethost -P $port –default-character-set=$charset <$filename 2. sqlserver: Note: sqlserver2003 and below only support osql, sqlserver2005 and above support both osql and sqlcmd. The following methods are common to both sqlcmd and osql. Parameter description: $user $passwrod $instance $dbname $filename $diff_filename Username Password Instance name Database name Full backup file name Incremental backup file name Full backup: osql -U $user -P “$password” -d master -S $instance -Q “ALTER DATABASE $dbname SET OFFLINE WITH ROLLBACK IMMEDIATE; RESTORE DATABASE $dbname FROM DISK='$filename& #39; WITH REPLACE; ALTER DATABASE $dbname SET ONLINE” Incremental backup: osql -U $user -P “$password” -d master -S $instance -Q “ALTER DATABASE $dbname SET OFFLINE WITH ROLLBACK IMMEDIATE; RESTORE DATABASE $dbname FROM DISK='$ filename' WITH REPLACE,NORECOVERY; RESTORE DATABASE $dbname FROM DISK='$diff_filename' WITH RECOVERY; ALTER DATABASE…
Install some PHP extensions geoip, mongo, redis, libevent, proctitl under debian
Because of the project, some new extensions need to be installed debian xfce 7.2 x64. The following three are all installed through apt-get. PHP 5.4.4-14deb7u5 (cli) nginx1.2.1 apache 2.2.22 1: Install through the default method (optimal). Most of the commonly used ones are in the debian warehouse and can be downloaded. Note that # represents root and can be passed through su- Due to the project, some new extensions need to be installed debian xfce 7.2 x64 The following three are installed through apt-get. PHP 5.4.4-14+deb7u5 (cli) nginx1.2.1 apache 2.2.22 1: Install through the default method (optimal). Most of the commonly used ones are in the debian warehouse and can be downloaded. Note that # stands for root. You can enter root through su – and enter the password. #apt-get install php5-geoip #apt-get install php-apc #apt-get install libevent-dev 2: Install through pecl (second) #pecl install mongo #pecl install channel://pecl.php.net/libevent-0.1.0 #pecl install channel://pecl.php.net/proctitle-0.1.2 #pecl install inotify #pecl install yaf After pecl is installed, you need to do several things. One is to add the corresponding so file to the corresponding ini, and the second is to link the ini to conf.d For example, the above three extensions, after pecl In debian…
2018.10.17HDFS, Hive, HBase, MongoDB, Redis, ES
HDFS: Hadoop’s distributed file system is called HDFS, which is a file system designed for storing very large files in a streaming data access mode. HDFS is suitable for: Storing and managing petabytes of data Processing unstructured data Focus on data processing throughput The application mode is: write once and read many times Not suitable for: Storing small files A large amount of randomness Need to modify files Hive: Hive is a data warehouse tool built on top of Hadoop. It can query and manage PB-level distributed data. It is suitable for For batch processing of large-scale data, Hive itself does not store data. It relies on HDFS and Mapreduce and provides a simple SQL-like query language HiveQL. However, there are some restrictions on using hive, and some standard sql does not exist in hive. The query speed is slow and does not provide real-time query and row-level data update operations. HBase: HBase is a distributed, column-oriented open source non-relational database (NoSQL) based on HDFS The file system supports efficient reading and writing at the record level, which is not suitable for offline analysis scenarios. Suitable for light query application scenarios with simple structure but very large data volume (usually…
Redis, MemCached, MongoDB Overview
I often hear and see what Redis is, what MemCached is… why is it used in my project? Because I wrote a Windows service, I need to keep reading the Sql Server 2008 R2 database. If I keep reading the database, Sql will go crazy! Solution: Interact with Redis first, and read Sql if successful. Greatly reduces the pressure on Sql. The research projects mainly include Redis, MemCached, MongoDB, and Amazon’s DynamoDB 1. Redis, MemCached, MongoDB Redis is an open source log-type Key-Value database written in ANSI C language, supports network, can be memory-based and persistent, and provides APIs in multiple languages. The development work is currently hosted by VMware. 1) Data ModelAs a Key-value database, Redis also provides the mapping relationship between keys and values. In addition to regular numerical values or strings, Redis key values can also be in one of the following forms:Lists (lists)Sets (sets)Sorted sets (ordered sets)Hashes (Hashes) Greek table)The data type of the key value determines the operations supported by the key value. Redis supports advanced atomic operations such as intersection, union, and difference of lists, sets, or ordered sets; at the same time, if the type of the key value is an ordinary number,…
About the relationship and role between mongodb, redis, and memcache.
Let me talk about my own use first: The first memcache is used, which is a server-side cache for key-value relationships. It is used to store some commonly used data that is not very large but requires quick response Then, in another place, I needed to use redis, and then I went to study redis. At first glance, it showed that I had installed the php extension. Because there is a redis server on the server, I did not install it locally. In fact, the usage It’s basically the same as memcache, except for a few parameters that are different. Of course, their caching effects are also different. What are the specific differences? Here are some information and my own summary 1. Redis and Memcache both store data in memory and are both in-memory databases. However, memcache can also be used to cache other things, such as images. , video, etc.2. Data type–Memcache must specify the byte length of the data when adding data, for example: set key3 0 0 8 lxsymcto STOREDAnd Redis is not required, such as: redis 127.0.0.1:6379>set key2 “lxsymblog” OK redis 127.0.0.1:6379>get key2 “lxsymblog”3. Virtual memory–Redis When the physical memory is used up, some values that…
Springboot integrates mybatis (mysql), mail, mongodb, cassandra, scheduler, redis, kafka, shiro, websocket
from:https://www.zybuluo.com/SailorXiao/note/136014 case online found that a machine’s memory load was very heavy. After top, it was found that a redis process occupied a large amount of memory. TOP … Recently I built a single-page website. All pages are loaded through partial refresh, without iframe, and we introduced dynamic tab: The content in all tabs is just one element, all on the same html page, without any iframe… 1. BOM hierarchy: window (can access BOM All elements in are the top-level elements) The next layer includes the following: →document (document’s attributes: forms.COOKIE.links/anchors. … The first one, the layout file created under the TabActivity solution, which contains the display components of multiple labels?xml version= 1.0 encoding= utf-8&qu… For each js Let’s make a summary analysis of various types of traversal, starting from the types and uses: array traversal and object traversal: as well as performance, advantages and disadvantages, etc. JS array traversal: 1. Ordinary for loop, commonly used array traversal var arr = [1,2,0 ,3,9]; for ( … A total of five steps are required: 1. Download the extension script 2. Cut the downloaded file to the fiddler installation directory 3. Restart fillder 4. Set up filtering Condition 5. Export the file…
