Archive

Author Archive

解决Python Error ‘TSaslClientTransport’ object has no attribute ‘trans’

November 12th, 2018 No comments

解决Python Error ‘TSaslClientTransport’ object has no attribute ‘trans’
原因应该是thrift和impyla包版本的问题

1
2
3
4
sudo pip uninstall thrift
sudo pip uninstall impyla
sudo pip install thrift==0.9.3
sudo pip install impyla==0.13.8

参考
http://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Python-Error-TSaslClientTransport-object-has-no-attribute/m-p/58033

Categories: 系统管理 Tags: ,

RQAlpha Docker容器化Dockerfile

November 12th, 2018 No comments

最近需要使用RQAlpha进行国内的量化分析工作,并对RQAlpha进行了容器化处理,
方便使用docker和ks进行容器化管理

1.基于 jupyter/minimal-notebook jupyter的官方镜像,默认使用python3
2.使用Anaconda的mini-conda进行环境管理,可以自由切换python27和python3运行环境
3.使用pip安装了其他非conda管理的包内容

1
pip install bs4 cx-Oracle docopt future hdfs pyecharts PyMySQL raven typing lxml

4.使用编译方式安装了TA-lib,暂时为32bit版本,安装了国内流行的tushare

1
pip install tushare TA-lib

5.将matplotlib.pyplot自动导入,同时配合安装了国内流行的pyechart
6.安装了HDFS,cx-Oracle,PyMySQL等常用的数据连接包

可以在以下位置找到
https://github.com/limccn/rqalpha/tree/master/docker

Read more…

Categories: 系统管理 Tags: ,

RQAlpha BUG Issue#219

November 12th, 2018 No comments

Pingback https://github.com/ricequant/rqalpha/issues/219

Hello, RQAlpha Team

RQAlpha is really a effective tool for price back-testing.

I found something wrong when using command `# rqalpha plot someresult.pkl` to plot my back-testing result. It came out a blank window. I tried to solve this problem and found something interesting.

in `rqalpha/rqalpha/mod/rqalpha_mod_sys_analyser/plot.py`, line 52-53

 portfolio = result_dict["portfolio"]
 benchmark_portfolio = result_dict.get("benchmark_portfolio")

 print portfolio.index
 print benchmark_portfolio.index

by printing portfolio.index and benchmark_portfolio.index , I found an unreasonable difference.

![image](https://user-images.githubusercontent.com/4476941/33165321-59a13c02-d071-11e7-80e1-b41a5fbdd6db.png)

According to https://github.com/pandas-dev/pandas/issues/8614 says, matplotlib can not plotting when DatetimeIndex is created by pandas > 0.15 .

I found a temporary way to solve this problem. and finally plotting was working functionally.

Use `index.to_pydatetime()` to explicitly convert `DatetimeIndex` type index to Python `Datetime` type.

For example: modify `rqalpha/rqalpha/mod/rqalpha_mod_sys_analyser/plot.py`, line 152
`ax.plot(portfolio[“unit_net_value”] – 1.0, label=_(u”strategy”), alpha=1, linewidth=2, color=red)`
to
`ax.plot(index.to_pydatetime(),portfolio[“unit_net_value”] – 1.0, label=_(u”strategy”), alpha=1, linewidth=2, color=red)`

Although it is not the best way to solve this problem, I know you can find the best one finally.
And I hope these information may help you.

Thanks.

Categories: 算法研究, 系统管理 Tags:

PostgreSQL查询表和index占用空间大小

November 12th, 2018 No comments

PostgreSQL查询表和index占用空间大小

PostgreSQL表和index占用空间大小信息存储在
information_schema.tables中
通过SQL可以查询到相应的统计数据

1
2
--查出单个表的大小
select pg_size_pretty(pg_relation_size('TABLENAME'));

查出表大小按大小含Index

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
-- 查出表大小按大小含Index
SELECT
"table_name",
pg_size_pretty(table_size) AS table_size,
pg_size_pretty(indexes_size) AS indexes_size,
pg_size_pretty(total_size) AS total_size
FROM (
SELECT
table_name,
SUBSTRING("table_name",1,10) as short_name,
pg_table_size(table_name) AS table_size,
pg_indexes_size(table_name) AS indexes_size,
pg_total_relation_size(table_name) AS total_size
FROM (
SELECT ('"' || table_schema || '"."' || table_name || '"') AS table_name
FROM information_schema.tables
) AS all_tables
where all_tables.table_name like '%TABLENAME%'
ORDER BY total_size DESC
) AS pretty_sizes

Read more…

Nginx HTTP站点使用301跳转HTTPS

November 12th, 2018 No comments

方法一,使用Return方式,适用于全站HTTPS

1
2
server_name  www.lidaren.com lidaren.com;
return       301 https://www.lidaren.com$request_uri;

方法二,rewirte方式转发特定目录,适用于子目录HTTPS化
location 使用 ·/· 根目录则全站跳转
location 使用 ·/XXX目录· 子目录则跳转指定子目录

1
2
3
location / {
    rewrite ^(.*) https://www.lidaren.com$1 permanent
}

完整参考

1
2
3
4
5
6
7
8
9
10
11
12
13
server {
    listen       80;
    server_name  www.lidaren.com lidaren.com;
 
# 方法一,使用Return方式,适用于全站HTTPS
    return       301 https://www.lidaren.com$request_uri;
 
# 方法二,rewirte方式转发特定目录,适用于子目录HTTPS化
# location 使用/根目录则全站跳转
#    location / {
#        rewrite ^(.*) https://www.lidaren.com$1 permanent
#    }
}
Categories: 系统管理, 零敲碎打 Tags:

Python 代码方式生产中国身份证号码

November 9th, 2018 No comments

Python 代码方式生产中国身份证号码
python做爬虫需要填写中国身份证号码,而且又各种验证规则,所以网上找了个靠谱的

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
import random, datetime
 
def ident_generator():
    #身份证号的前两位,省份代号
    sheng = ('11', '12', '13', '14', '15', '21', '22', '23', '31', '32', '33', '34', '35', '36', '37', '41', '42', '43', '44', '45', '46', '50', '51', '52', '53', '54', '61', '62', '63', '64', '65', '66')
 
    #随机选择距离今天在7000到25000的日期作为出生日期(没有特殊要求我就随便设置的,有特殊要求的此处可以完善下)
    birthdate = (datetime.date.today() - datetime.timedelta(days = random.randint(7000, 25000)))
 
    #拼接出身份证号的前17位(第3-第6位为市和区的代码,中国太大此处就偷懒了写了定值,有要求的可以做个随机来完善下;第15-第17位为出生的顺序码,随机在100到199中选择)
    ident = sheng[random.randint(0, 31)] + '0101' + birthdate.strftime("%Y%m%d") + str(random.randint(100, 199))
 
    #前17位每位需要乘上的系数,用字典表示,比如第一位需要乘上7,最后一位需要乘上2
    coe = {1: 7, 2: 9, 3: 10, 4: 5, 5: 8, 6: 4, 7: 2, 8: 1, 9: 6, 10: 3, 11:7, 12: 9, 13: 10, 14: 5, 15: 8, 16: 4, 17: 2}
    summation = 0
 
    #for循环计算前17位每位乘上系数之后的和
    for i in range(17):
        summation = summation + int(ident[i:i + 1]) * coe[i+1]#ident[i:i+1]使用的是python的切片获得每位数字
 
    #前17位每位乘上系数之后的和除以11得到的余数对照表,比如余数是0,那第18位就是1
    key = {0: '1', 1: '0', 2: 'X', 3: '9', 4: '8', 5: '7', 6: '6', 7: '5', 8: '4', 9: '3', 10: '2'}
 
    #拼接得到完整的18位身份证号
    return ident + key[summation % 11]
 
ident_generator()

http://www.51testing.com/html/12/15124112-3705453.html

Categories: 语言编程, 零敲碎打 Tags:

npm中Error: could not get uid/gid问题的解决方法

November 9th, 2018 No comments

在Docker中运行npm出现Error: could not get uid/gid的问题

但是通过设置`unsafe-perm true` 可以解决这个问题,不可思议。

1
npm config set unsafe-perm true

其他可以用的npm配置
# 关闭安全证书检查

1
npm config set unsafe-perm true

# 关闭强制SSL

1
npm config set strict-ssl false

# 更改npm源

1
npm config set registry https://registry.npm.taobao.org

#设置代理

1
2
3
4
# socks5 proxy
npm config set proxy http://xx@xx.com:xxxx 
# http proxy
npm config set https-proxy=http://xx@xx.com:xxxx

参考
http://www.cnblogs.com/liyongjian5179/p/9884944.html

Categories: 系统管理 Tags: ,