Archive

Archive for the ‘系统管理’ Category

解决Python Error ‘TSaslClientTransport’ object has no attribute ‘trans’

November 12th, 2018 No comments

解决Python Error ‘TSaslClientTransport’ object has no attribute ‘trans’
原因应该是thrift和impyla包版本的问题

1
2
3
4
sudo pip uninstall thrift
sudo pip uninstall impyla
sudo pip install thrift==0.9.3
sudo pip install impyla==0.13.8

参考
http://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Python-Error-TSaslClientTransport-object-has-no-attribute/m-p/58033

Categories: 系统管理 Tags: ,

RQAlpha Docker容器化Dockerfile

November 12th, 2018 No comments

最近需要使用RQAlpha进行国内的量化分析工作,并对RQAlpha进行了容器化处理,
方便使用docker和ks进行容器化管理

1.基于 jupyter/minimal-notebook jupyter的官方镜像,默认使用python3
2.使用Anaconda的mini-conda进行环境管理,可以自由切换python27和python3运行环境
3.使用pip安装了其他非conda管理的包内容

1
pip install bs4 cx-Oracle docopt future hdfs pyecharts PyMySQL raven typing lxml

4.使用编译方式安装了TA-lib,暂时为32bit版本,安装了国内流行的tushare

1
pip install tushare TA-lib

5.将matplotlib.pyplot自动导入,同时配合安装了国内流行的pyechart
6.安装了HDFS,cx-Oracle,PyMySQL等常用的数据连接包

可以在以下位置找到
https://github.com/limccn/rqalpha/tree/master/docker

Read more…

Categories: 系统管理 Tags: ,

RQAlpha BUG Issue#219

November 12th, 2018 No comments

Pingback https://github.com/ricequant/rqalpha/issues/219

Hello, RQAlpha Team

RQAlpha is really a effective tool for price back-testing.

I found something wrong when using command `# rqalpha plot someresult.pkl` to plot my back-testing result. It came out a blank window. I tried to solve this problem and found something interesting.

in `rqalpha/rqalpha/mod/rqalpha_mod_sys_analyser/plot.py`, line 52-53

 portfolio = result_dict["portfolio"]
 benchmark_portfolio = result_dict.get("benchmark_portfolio")

 print portfolio.index
 print benchmark_portfolio.index

by printing portfolio.index and benchmark_portfolio.index , I found an unreasonable difference.

![image](https://user-images.githubusercontent.com/4476941/33165321-59a13c02-d071-11e7-80e1-b41a5fbdd6db.png)

According to https://github.com/pandas-dev/pandas/issues/8614 says, matplotlib can not plotting when DatetimeIndex is created by pandas > 0.15 .

I found a temporary way to solve this problem. and finally plotting was working functionally.

Use `index.to_pydatetime()` to explicitly convert `DatetimeIndex` type index to Python `Datetime` type.

For example: modify `rqalpha/rqalpha/mod/rqalpha_mod_sys_analyser/plot.py`, line 152
`ax.plot(portfolio[“unit_net_value”] – 1.0, label=_(u”strategy”), alpha=1, linewidth=2, color=red)`
to
`ax.plot(index.to_pydatetime(),portfolio[“unit_net_value”] – 1.0, label=_(u”strategy”), alpha=1, linewidth=2, color=red)`

Although it is not the best way to solve this problem, I know you can find the best one finally.
And I hope these information may help you.

Thanks.

Categories: 算法研究, 系统管理 Tags:

PostgreSQL查询表和index占用空间大小

November 12th, 2018 No comments

PostgreSQL查询表和index占用空间大小

PostgreSQL表和index占用空间大小信息存储在
information_schema.tables中
通过SQL可以查询到相应的统计数据

1
2
--查出单个表的大小
select pg_size_pretty(pg_relation_size('TABLENAME'));

查出表大小按大小含Index

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
-- 查出表大小按大小含Index
SELECT
"table_name",
pg_size_pretty(table_size) AS table_size,
pg_size_pretty(indexes_size) AS indexes_size,
pg_size_pretty(total_size) AS total_size
FROM (
SELECT
table_name,
SUBSTRING("table_name",1,10) as short_name,
pg_table_size(table_name) AS table_size,
pg_indexes_size(table_name) AS indexes_size,
pg_total_relation_size(table_name) AS total_size
FROM (
SELECT ('"' || table_schema || '"."' || table_name || '"') AS table_name
FROM information_schema.tables
) AS all_tables
where all_tables.table_name like '%TABLENAME%'
ORDER BY total_size DESC
) AS pretty_sizes

Read more…

Nginx HTTP站点使用301跳转HTTPS

November 12th, 2018 No comments

方法一,使用Return方式,适用于全站HTTPS

1
2
server_name  www.lidaren.com lidaren.com;
return       301 https://www.lidaren.com$request_uri;

方法二,rewirte方式转发特定目录,适用于子目录HTTPS化
location 使用 ·/· 根目录则全站跳转
location 使用 ·/XXX目录· 子目录则跳转指定子目录

1
2
3
location / {
    rewrite ^(.*) https://www.lidaren.com$1 permanent
}

完整参考

1
2
3
4
5
6
7
8
9
10
11
12
13
server {
    listen       80;
    server_name  www.lidaren.com lidaren.com;
 
# 方法一,使用Return方式,适用于全站HTTPS
    return       301 https://www.lidaren.com$request_uri;
 
# 方法二,rewirte方式转发特定目录,适用于子目录HTTPS化
# location 使用/根目录则全站跳转
#    location / {
#        rewrite ^(.*) https://www.lidaren.com$1 permanent
#    }
}
Categories: 系统管理, 零敲碎打 Tags:

npm中Error: could not get uid/gid问题的解决方法

November 9th, 2018 No comments

在Docker中运行npm出现Error: could not get uid/gid的问题

但是通过设置`unsafe-perm true` 可以解决这个问题,不可思议。

1
npm config set unsafe-perm true

其他可以用的npm配置
# 关闭安全证书检查

1
npm config set unsafe-perm true

# 关闭强制SSL

1
npm config set strict-ssl false

# 更改npm源

1
npm config set registry https://registry.npm.taobao.org

#设置代理

1
2
3
4
# socks5 proxy
npm config set proxy http://xx@xx.com:xxxx 
# http proxy
npm config set https-proxy=http://xx@xx.com:xxxx

参考
http://www.cnblogs.com/liyongjian5179/p/9884944.html

Categories: 系统管理 Tags: ,

Linux使用sed命令对进行文件行操作

November 9th, 2018 No comments

1、删除文档的第一行
sed -i ‘1d’
2、删除文档的最后一行
sed -i ‘$d’
3、在文档指定行中增加一行
例如文档如下:
echo “1”;
echo “2”;
echo “4”;
echo “5”;
想要在echo “2”;后面加上一条echo “3”;可以用如下命令
sed -i ‘/echo “2”;/aecho “3”;’
之所以用分号,是因为文本中本来就有。也就是说分号不是必须的!
抽象出来就是: sed -i ‘/* /a*’
4、删除文件中的一行
sed -i ‘3d’
5、删除文件中包含某个关键字开头的所有行
sed -i ‘/^QWQ/d’
6、删除文件中包含某个关键字的所有行
sed -i ‘/QWQ/d’

Categories: 系统管理 Tags: ,