Overview: Perbedaan revisi
(→Access security) |
|||
| Baris 53: | Baris 53: | ||
==[[IT:Queues|Queues]]== <!--T:15--> | ==[[IT:Queues|Queues]]== <!--T:15--> | ||
| − | The AWS Simple Queue System (SQS) is used to manage the exchange of tasks between the different servers in the VPC. For example, an incoming email is analysed by the messaging server, and then put in the processing queue. The processing server then gets the request | + | The AWS Simple Queue System (SQS) is used to manage the exchange of tasks between the different servers in the VPC. For example, an incoming email is analysed by the messaging server, and then put in the processing queue. The processing server then gets the request from the queue and processes it. Once finished, if there is a response email to send, a request is put on the outgoing email queue, and if there is a report to be generated, a request is put on the reporting queue, to be picked up by the next available report server. |
This approach, known as 'loose coupling', allows complex and processor-intensive tasks to be shared between a cluster of different servers, increasing speed and redundancy. | This approach, known as 'loose coupling', allows complex and processor-intensive tasks to be shared between a cluster of different servers, increasing speed and redundancy. | ||
Revisi per 22 Juni 2018 16.47
Daftar isi
iSIKHNAS Server Infrastructure
iSIKHNAS is hosted in a Virtual Private Cloud (VPC) on Amazon Web Services (AWS) cloud servers. The VPC consists of a number of different components (servers, queues, storage) that expand and contract according to need.
Security
Access security
The Virtual Private Cloud allows computers within the cloud to communicate between themselves rapidly and easily, while blocking all access from outside the VPC. Access is controlled by a single, highly secure NAT server.
All communications with iSIKHNAS are encrypted using secure industry standard Transport Layer Security (TLS) encryption, preventing interception of any data exchanged between the user and the VPC.
Administrative access through the NAT server is controlled using digital certificates.
Data security and redundancy
The iSIKHNAS database runs on a single master database, with multiple slave databases providing real-time replication. Some of the slaves are always in a different physical location to the master. In case of failure of the master, one of the slaves is automatically promoted to be the master, resulting in down-time of only a few seconds.
In addition to database replication, the full system is automatically backed up every 12 hours and stored in replicated off-site storage.
The VPC is spread across two security zones, representing different physical locations, with servers and queues replicated in each zone. This means that should there be a catastrophic failure of an entire data centre, the system will continue to run uninterrupted.
Servers
The VPC consists of a number of different servers dedicated to different tasks:
- NAT (for controlling access to the VPC)
- Database (master and read-replica slave servers)
- Web servers (for hosting the web site)
- Messaging server (for managing SMS, IM and email)
- Processing (for processing incoming requests on email)
- Reporting (for analysing data and generating outputs)
- Wiki (for managing this Wiki server)
While some of these servers are single fixed instances, others are Elastic Cloud Computing (EC2) instances. This means that, as demand increases, the server is automatically replicated to increase capacity. As demand decreases, servers no longer in use are shut down.
All servers use the Ubuntu operating system, with open source and custom built software configured to optimise the performance of their particular tasks.
Storage
AWS S3 storage buckets are used to provide rapid access to unlimited storage for both temporary and static data. The content of each bucket is automatically replicated and stored in three different physical locations.
Queues
The AWS Simple Queue System (SQS) is used to manage the exchange of tasks between the different servers in the VPC. For example, an incoming email is analysed by the messaging server, and then put in the processing queue. The processing server then gets the request from the queue and processes it. Once finished, if there is a response email to send, a request is put on the outgoing email queue, and if there is a report to be generated, a request is put on the reporting queue, to be picked up by the next available report server. This approach, known as 'loose coupling', allows complex and processor-intensive tasks to be shared between a cluster of different servers, increasing speed and redundancy.
Daemons
In order to respond to various system events, a number of daemons have been created. These are small programs that are running full time in the background, waiting for a job. When an event occurs (such as a new IM message, or a message is available on a queue that the daemon serves), the daemon processes that message, and continues to wait.
iSIKHNAS daemons are written in Python, and controlled with the Linux upstart tool, which manages starting, stopping, and automatically re-starting the daemon in case of failure.
Daemons include:
- Processor for incoming email
- Report processor
- Instant messaging system
- E-mail sender
Technologies
iSIKHNAS uses a range of AWS technologies to maximise robustness, performance, security and flexibility. These include:
- EC2 (Elastic cloud computing for cost-effective response to demand)
- Route 53 (scalable domain name system)
- VPC (Virtual private cloud, for security)
- S3 (Scalable storage)
- IAM (secure access control)
- SES (e-mail sending service)
- SQS (messaging queue service)
- CloudFront (cached content delivery for increased speed and efficiency)
Software
iSIKHNAS uses a range of open source and custom-built software, on Ubuntu servers. The main tools used are:
- Apache2 web server
- PostgreSQL database server, with the PostGIS extension
- Python programming language
- PHP and the Yii framework for web page deliver
- MediaWiki for the Wiki