Archive

Archive for the ‘系统管理’ Category

nohup 后台运行python程序print无输出

July 26th, 2019 No comments

使用nohup后台运行python,print没有输出到日志

nohup python foobar.py > foobar.log 2>&1 &

发现foobar.log中显示不出来python程序中print的东西。
这是因为python的输出有缓冲,导致foobar.log并不能够马上看到输出。
python 有个-u参数,使得python不启用缓冲。

nohup python -u foobar.py > foobar.log 2>&1 &

其他玩法:
只输出错误到日志
# Only error write to log
nohup python -u ./foobar.py> /dev/null 2>foobar.log &
不输出到日志
# Nothing to Display
nohup python -u ./foobar.py> /dev/null 2>&1 &
全部print输出到日志
# Write all to log
nohup python -u ./foobar.py> ./foobar.log 2>&1 &

https://blog.csdn.net/Statham_stone/article/details/78290813

Categories: 系统管理 Tags:

解决Python Error ‘TSaslClientTransport’ object has no attribute ‘trans’

November 12th, 2018 No comments

解决Python Error ‘TSaslClientTransport’ object has no attribute ‘trans’
原因应该是thrift和impyla包版本的问题

sudo pip uninstall thrift
sudo pip uninstall impyla
sudo pip install thrift==0.9.3
sudo pip install impyla==0.13.8

参考
http://community.cloudera.com/t5/Interactive-Short-cycle-SQL/Python-Error-TSaslClientTransport-object-has-no-attribute/m-p/58033

Categories: 系统管理 Tags: ,

RQAlpha Docker容器化Dockerfile

November 12th, 2018 No comments

最近需要使用RQAlpha进行国内的量化分析工作,并对RQAlpha进行了容器化处理,
方便使用docker和ks进行容器化管理

1.基于 jupyter/minimal-notebook jupyter的官方镜像,默认使用python3
2.使用Anaconda的mini-conda进行环境管理,可以自由切换python27和python3运行环境
3.使用pip安装了其他非conda管理的包内容

pip install bs4 cx-Oracle docopt future hdfs pyecharts PyMySQL raven typing lxml

4.使用编译方式安装了TA-lib,暂时为32bit版本,安装了国内流行的tushare

pip install tushare TA-lib

5.将matplotlib.pyplot自动导入,同时配合安装了国内流行的pyechart
6.安装了HDFS,cx-Oracle,PyMySQL等常用的数据连接包

可以在以下位置找到
https://github.com/limccn/rqalpha/tree/master/docker

Read more…

Categories: 系统管理 Tags: ,

RQAlpha BUG Issue#219

November 12th, 2018 No comments

Pingback https://github.com/ricequant/rqalpha/issues/219

Hello, RQAlpha Team

RQAlpha is really a effective tool for price back-testing.

I found something wrong when using command `# rqalpha plot someresult.pkl` to plot my back-testing result. It came out a blank window. I tried to solve this problem and found something interesting.

in `rqalpha/rqalpha/mod/rqalpha_mod_sys_analyser/plot.py`, line 52-53

 portfolio = result_dict["portfolio"]
 benchmark_portfolio = result_dict.get("benchmark_portfolio")

 print portfolio.index
 print benchmark_portfolio.index

by printing portfolio.index and benchmark_portfolio.index , I found an unreasonable difference.

![image](https://user-images.githubusercontent.com/4476941/33165321-59a13c02-d071-11e7-80e1-b41a5fbdd6db.png)

According to https://github.com/pandas-dev/pandas/issues/8614 says, matplotlib can not plotting when DatetimeIndex is created by pandas > 0.15 .

I found a temporary way to solve this problem. and finally plotting was working functionally.

Use `index.to_pydatetime()` to explicitly convert `DatetimeIndex` type index to Python `Datetime` type.

For example: modify `rqalpha/rqalpha/mod/rqalpha_mod_sys_analyser/plot.py`, line 152
`ax.plot(portfolio[“unit_net_value”] – 1.0, label=_(u”strategy”), alpha=1, linewidth=2, color=red)`
to
`ax.plot(index.to_pydatetime(),portfolio[“unit_net_value”] – 1.0, label=_(u”strategy”), alpha=1, linewidth=2, color=red)`

Although it is not the best way to solve this problem, I know you can find the best one finally.
And I hope these information may help you.

Thanks.

Categories: 算法研究, 系统管理 Tags:

PostgreSQL查询表和index占用空间大小

November 12th, 2018 No comments

PostgreSQL查询表和index占用空间大小

PostgreSQL表和index占用空间大小信息存储在
information_schema.tables中
通过SQL可以查询到相应的统计数据

--查出单个表的大小
SELECT pg_size_pretty(pg_relation_size('TABLENAME'));

查出表大小按大小含Index

-- 查出表大小按大小含Index
SELECT
"table_name",
pg_size_pretty(table_size) AS table_size,
pg_size_pretty(indexes_size) AS indexes_size,
pg_size_pretty(total_size) AS total_size
FROM (
SELECT
TABLE_NAME,
SUBSTRING("table_name",1,10) AS short_name,
pg_table_size(TABLE_NAME) AS table_size,
pg_indexes_size(TABLE_NAME) AS indexes_size,
pg_total_relation_size(TABLE_NAME) AS total_size
FROM (
SELECT ('"' || table_schema || '"."' || TABLE_NAME || '"') AS TABLE_NAME
FROM information_schema.tables
) AS all_tables
WHERE all_tables.table_name LIKE '%TABLENAME%'
ORDER BY total_size DESC
) AS pretty_sizes

Read more…

Nginx HTTP站点使用301跳转HTTPS

November 12th, 2018 No comments

方法一,使用Return方式,适用于全站HTTPS

server_name  www.lidaren.com lidaren.com;
return       301 https://www.lidaren.com$request_uri;

方法二,rewirte方式转发特定目录,适用于子目录HTTPS化
location 使用 ·/· 根目录则全站跳转
location 使用 ·/XXX目录· 子目录则跳转指定子目录

location / {
    rewrite ^(.*) https://www.lidaren.com$1 permanent
}

完整参考

server {
    listen       80;
    server_name  www.lidaren.com lidaren.com;
 
# 方法一,使用Return方式,适用于全站HTTPS
    return       301 https://www.lidaren.com$request_uri;
 
# 方法二,rewirte方式转发特定目录,适用于子目录HTTPS化
# location 使用/根目录则全站跳转
#    location / {
#        rewrite ^(.*) https://www.lidaren.com$1 permanent
#    }
}
Categories: 系统管理, 零敲碎打 Tags:

npm中Error: could not get uid/gid问题的解决方法

November 9th, 2018 No comments

在Docker中运行npm出现Error: could not get uid/gid的问题

但是通过设置`unsafe-perm true` 可以解决这个问题,不可思议。

npm config set unsafe-perm true

其他可以用的npm配置
# 关闭安全证书检查

npm config set unsafe-perm true

# 关闭强制SSL

npm config set strict-ssl false

# 更改npm源

npm config set registry https://registry.npm.taobao.org

#设置代理

# socks5 proxy
npm config set proxy http://xx@xx.com:xxxx 
# http proxy
npm config set https-proxy=http://xx@xx.com:xxxx

参考
http://www.cnblogs.com/liyongjian5179/p/9884944.html

Categories: 系统管理 Tags: ,