Perl & Python编写CGI
近期偶然玩了一下CGI，收集点资料写篇在这里留档。
如今想做HTTP Cache回归測试了，为了模拟不同的响应头及数据大小。就须要一个CGI按须要传回指定的响应头和内容。这是从老外的測试页面学习到的经验。
CGI事实上就是使用STDIN和环境变量作为输入, STDOUT做为输出。按照Http协议生成相应的数据。
一. 数据输出
数据输出遵循HTTP协议，分为四部分:
状态行 (Status Line):
200 OK
响应头(Response Headers):
Content-Type: text/html
Content-Length: 3072
空白行(代表响应头已经列完, 仅仅能包括回车和换行符):
数据(文本或二进制数据):
<html>xxxxxx</html>
參考: HTTP Protocol
状态行和响应头中每一行都必须以\r\n(回车及换行)结束。使用这个规则，以简单的文本组装就能够完毕响应了。
仅仅要注意二进制数据输出方法就能够了。 尝试设定Content-Length时可能没办法设定的准确，至少对于Apache Server是这样，只是不影响最后返回的数据量。
二. 数据输入
CGI的数据输入主要是环境变量，以下这个链接有一串定义,
Key | Value |
DOCUMENT_ROOT | The root directory of your server |
HTTP_COOKIE | The visitor's cookie, if one is set |
HTTP_HOST | The hostname of the page being attempted |
HTTP_REFERER | The URL of the page that called your program |
HTTP_USER_AGENT | The browser type of the visitor |
HTTPS | "on" if the program is being called through a secure server |
PATH | The system path your server is running under |
QUERY_STRING | The query string (see GET, below) |
REMOTE_ADDR | The IP address of the visitor |
REMOTE_HOST | The hostname of the visitor (if your server has reverse-name-lookups on; otherwise this is the IP address again) |
REMOTE_PORT | The port the visitor is connected to on the web server |
REMOTE_USER | The visitor's username (for .htaccess-protected pages) |
REQUEST_METHOD | GET or POST |
REQUEST_URI | The interpreted pathname of the requested document or CGI (relative to the document root) |
SCRIPT_FILENAME | The full pathname of the current CGI |
SCRIPT_NAME | The interpreted pathname of the current CGI (relative to the document root) |
SERVER_ADMIN | The email address for your server's webmaster |
SERVER_NAME | Your server's fully qualified domain name (e.g. www.cgi101.com) |
SERVER_PORT | The port number your server is listening on |
SERVER_SOFTWARE | The server software you're using (e.g. Apache 1.3) |
当你须要CGI处理POST请求时，CGI就要使用STDIN做为输入来接收数据了。
在Perl里使用以下的代码读取:
use CGI;
my $cgi = CGI->new();
my %params = $cgi->Vars();
而在Python则是:
import cgi
cgi.FieldStorage()
三. 开发语言
能够写CGI的开发语言太多，以下附上两个分别使用Perl和Python编写的同样功能的CGI, 正好能够做个对照。
这两个脚本能够接收Query String, 然后返回不同的文件。数据大小，以及缓存相关的头信息，能够区分二进制数据和文本数据。
CGI脚本支持以Query String改动以下响应头:
content type,
cache control,
content length (仅仅返回相应大小的数据),
last modified,
expires
以下是返回一个指定大小图片的样例:
/cgi/cache_factory.pl?type=image&size=32&last-modified=Fri, 02 Apr 2014 02:34:06 GMT&cache-control=private,max-age=60&expires=Fri, 22 Apr 2014 02:34:06 GMT
代码非常easy。能够做个參考。
Perl版本号
#!/usr/bin/perl
use strict;
use warnings;
use CGI;
use constant BUFFER_SIZE => 4_096;
use constant DATA_DIRECTORY => "/var/www/Cache";
my $cgi = CGI->new();
my %params = $cgi->Vars();
&parserCommonHeaders;
if(exists $params{'type'} && $params{'type'}=="image"){
&generateImageData;
}
else{
&generateTextData;
}
sub parserCommonHeaders{
if(exists $params{'cache-control'}){
print 'Cache-Control:',$params{'cache-control'},"\r\n";
}
if(exists $params{'last-modified'}){
print 'Last-Modified:',$params{'last-modified'},"\r\n";
}
if(exists $params{'expires'}){
print 'Expires:',$params{'expires'},"\r\n";
}
if(exists $params{'etag'}){
print 'ETag:ea6186e11526ce1:0',"\r\n";
}
}
sub generateImageData{
my $buffer = "";
my $targetSize = 100*1024*1024;
if(exists $params{'size'} && $params{'size'}>0){
$targetSize = 1024*$params{'size'};
print "Content-length: $targetSize \r\n";
}
my $image = DATA_DIRECTORY .'/images/very_big.jpg';
my( $type ) = $image =~ /\.(\w+)$/;
$type eq "jpg" and $type == "jpeg";
print $cgi->header( -type => "image/$type", -expires => "-1d" );
binmode STDOUT;
local *IMAGE;
open IMAGE, $image or die "Cannot open file $image: $!";
my $sentSize = 0;
while ( read( IMAGE, $buffer, BUFFER_SIZE ) ) {
print $buffer;
$sentSize += BUFFER_SIZE;
if($sentSize>=$targetSize){
last;
}
}
close IMAGE;
}
sub generateTextData{
my $startHeader = '<html><head><title>HTTP Cache Testing HTML</title></head><body>';
my $tailPart = '</body></html>';
if(exists $params{'type'}){
print "Content-type:$params{'type'}\r\n";
}
else{
print "Content-type:text/html\r\n";
}
my $targetTextSize = 100*1024*1024;
if(exists $params{'size'} && $params{'size'}>0){
$targetTextSize = 1024*$params{'size'};
print "Content-length: $targetTextSize \r\n";
}
print "\r\n";
$targetTextSize -= length($startHeader) + length($tailPart);
print "$startHeader";
my $filepath = DATA_DIRECTORY .'/files/big_text.txt';
open(FILE, $filepath) or die $!;
my @lines = <FILE>;
close(FILE);
foreach my $line (@lines) {
if( length($line)<=$targetTextSize ){
print $line;
}
else{
print substr($line,0,$targetTextSize);
}
$targetTextSize -= length($line);
if($targetTextSize<=0){
last;
}
}
print "$tailPart";
}
Python版本号
#!/usr/bin/python
import os
import cgi
import sys
BUFFER_SIZE = 4096
DATA_DIRECTORY = "/var/www/Cache"
def parserCommonHeaders(formQuery):
if('cache-control' in formQuery.keys()):
print 'Cache-Control:',formQuery['cache-control'].value,"\r\n",
if('last-modified' in formQuery.keys()):
print 'Last-Modified:',formQuery['last-modified'].value,"\r\n",
if('expires' in formQuery.keys()):
print 'Expires:',formQuery['expires'].value,"\r\n",
if('etag' in formQuery.keys()):
print 'ETag:ea6186e11526ce1:0',"\r\n",
def generateImageData(formQuery):
targetSize = 100*1024*1024;
if('size' in formQuery.keys()):
targetSize = 1024*int(formQuery['size'].value)
print "Content-length:",targetSize,"\r\n",
image = DATA_DIRECTORY+'/images/very_big.jpg'
print "Content-Type:image/jpeg\r\n",
print
sentSize = 0
f = open(image, 'rb')
while True:
data = f.read(4096)
sentSize = sentSize + BUFFER_SIZE
sys.stdout.write(data)
if sentSize>targetSize:
break
sys.stdout.flush()
close(f)
def generateTextData(formQuery):
startHeader = '<html><head><title>HTTP Cache Testing HTML</title></head><body>'
tailPart = '</body></html>'
targetTextSize = 2.3*1024*1024;
if('size' in formQuery.keys()):
targetTextSize = 1024*int(formQuery['size'].value)
print "Content-length:",targetTextSize,"\r\n",
if('type' in formQuery.keys()):
print "Content-type: %s\r\n"%(formQuery['type'].value),
else:
print "Content-type: text/html\r\n",
print
print startHeader
targetTextSize = targetTextSize - len(startHeader) - len(tailPart)
filepath = DATA_DIRECTORY + '/files/big_text.txt'
file = open(filepath)
lines = file.readlines()
file.close()
for line in lines:
if( len(line) <= targetTextSize ):
print line
else:
print line[0:targetTextSize]
targetTextSize = targetTextSize - len(line)
if(targetTextSize<=0):
break
print tailPart
if __name__ =="__main__":
formQuery = cgi.FieldStorage() #os.environ['QUERY_STRING']
parserCommonHeaders(formQuery)
if('type' in formQuery.keys() and formQuery['type'].value=="image"):
generateImageData(formQuery)
else:
generateTextData(formQuery)
四. 服务器
服务器端使用Apache Server+WANem, 配合CGI完毕灵活的需求，开通SFTP端口供相关同学编辑。方便共享測试用例。
.-----. .-------.
| CGI | | WANem |
'-----'---'-------'
| Apache Server |
'-----------------'
^
|
SFTP & HTTP
|
.------------------.
| Web Page Editor |
| and Browser |
'------------------'
配有WANem最大的优点就是能够依据需求进网络状态调整, 甚至能够用相似以下的方式在測试用例动态调整。
參考:
Apache配置 (不要忘记给CGI脚本加上可运行权限)