Save
Saving
  • guluting

    启动容器命令:docker start d21a

    报错

    Error response from daemon: devmapper: Error mounting '/dev/mapper/docker-253:3-537354454-cdb4e31a34f55981f2cabe775700719ecc20bfdc6b85c1b26245b41026daa8fe' on '/var/lib/docker/devicemapper/mnt/cdb4e31a34f55981f2cabe775700719ecc20bfdc6b85c1b26245b41026daa8fe': invalid argument
    

    解决方式:
    /var/lib/docker/containers/d21a*目录中找到config.v2.json,然后vi config.v2.json
    找到MountLabelProcessLabel个键,并把其对应的引号中的内容清空即可

    posted in GuaiK机房 read more
  • guluting

    一、安装及测试

    1、安装

    useradd -s /sbin/nologin -d /opt/minio minio
    mkdir -p /opt/minio/bin
    mkdir /opt/minio/data
    wget https://dl.minio.io/server/minio/release/linux-amd64/minio -O /opt/minio/bin/minio
    chmod +x /opt/minio/bin/minio
    chown -R minio:minio /opt/minio
    

    2、添加配置文件

    vi /opt/minio/minio.conf

    MINIO_VOLUMES=/opt/minio/data
    MINIO_OPTS="--address :9090"
    MINIO_ACCESS_KEY=[YOUR_ACCESS_KEY]
    MINIO_SECRET_KEY=[YOUR_SECRET_KEY]
    

    3、添加系统服务

    vi /etc/systemd/system/minio.service

    [Unit]
    Description=Minio
    Documentation=https://docs.minio.io
    Wants=network-online.target
    After=network-online.target
    AssertFileIsExecutable=/opt/minio/bin/minio
    
    [Service]
    WorkingDirectory=/opt/minio
    
    User=minio
    Group=minio
    
    PermissionsStartOnly=true
    
    EnvironmentFile=-/opt/minio/minio.conf
    ExecStartPre=/bin/bash -c "[ -n \"${MINIO_VOLUMES}\" ] || echo \"Variable MINIO_VOLUMES not set in /opt/minio/minio.conf\""
    
    ExecStart=/opt/minio/bin/minio server $MINIO_OPTS $MINIO_VOLUMES
    
    StandardOutput=journal
    StandardError=inherit
    
    # Specifies the maximum file descriptor number that can be opened by this process
    LimitNOFILE=65536
    
    # Disable timeout logic and wait until process is stopped
    TimeoutStopSec=0
    
    # SIGTERM signal is used to stop Minio
    KillSignal=SIGTERM
    
    SendSIGKILL=no
    
    SuccessExitStatus=0
    
    [Install]
    WantedBy=multi-user.target
    

    4、启动Minio服务

    systemctl enable minio
    systemctl start minio
    systemctl status minio
    

    PS:默认minio的监听端口是:9000,我在配置文件中改成了9090,所以可以通过:
    http://host:9090访问minio后台。

    posted in GuaiK机房 read more
  • guluting

    如果是Key-Vlue类型的数据(dict,map)相关的,你默认它遍历的顺序是随机的就好啦,它们本身就是无序的;有序的话主要还是List。

    posted in GuaiK故障处理 read more
  • guluting

    📣🛠 为了确保《怪客实验室》的数据安全,以及国外主机的各种不稳定因素。本站会不定期的关闭并备份所有数据库及文件资料。

    posted in GuaiK新闻 read more
  • guluting

    # -*- coding:utf-8 -*-
    # 作者:luting.gu@gmail.com
    # 倒排索引-搜索引擎 DEMO
    import sys
    reload(sys)
    sys.setdefaultencoding('utf-8')
    
    # t1到t7都是待搜索的全文
    t1 = """
          在我们家族,各行各业从业者都有。从政的,从商的,人民教师的,农民的,企业职员的。从收入水平看,就目前而言,自然是从商的赚的钱稍微比其他人都多一些。但是,亲眼目睹过去几年几位兄弟从商的工作状态,我真的没有羡慕过他们收入比我高,因为,他们的那种工作强度,那种紧张忙碌,那种工作压力,是我根本承受不起的。
    """
    t2 = """
    处女座为了爱情付出的不是金钱又或者时间之类的东西,而是心态,她们的思维模式比较单一,向往平凡的爱情模式,对于金钱的向往是从来没有过的,两个人在一起开开心心就好,心愿纯真简单。但是如果遇到渣男,她就会患得患失的思考为什么会这样,会造成她经常纠结是否是自身的问题,不断否定自己,想不开,久而久之很容易得与抑郁症。
    """
    t3 = """
    狮子座天生的骄傲会自豪,随着她的恋情一步一步的消失殆尽,越来越迁就恋人,狮子认为在生命中如果没有一次奋不顾身的恋爱,是不完整的。可是一旦恋情失败,狮子会变成另一个极端的自己,越发的讨厌以前的自己,抛弃原来的自我。或许有一天,她遇见对的人,会再次重生吧。
    """
    t4 = """
    人心有真假,时间能见证,感情有冷暖,风雨能考验,时间带不走真正的朋友,岁月留不住虚幻的拥有,时光转换,体会到缘分善变,平淡无语,感受了人情冷暖,有心的人,不管你在与不在,都会惦念,无心的情,无论你好与不好,只是漠然,走过一段路,总能有一次领悟,经历一些事,才能看清一些人。
    """
    t5 = """
    总会有些不喜欢,又不得不去做的事,与其每次抱着自杀的心情去面对,不如把它们养成习惯。习惯跑步,习惯健身,习惯记录,习惯和一个人在一起。习惯能消磨一个人对事物的厌恶感,能让一个人与世界相安无事。“习惯了就好”不是无奈,而是学会了对自己体谅。—— 刘同
    """
    t6 = """
    想知道一个人爱不爱你,就给他自由。不要妄图拴住爱情,世上没有能拴得住的爱人。若是爱你,何需拴着他?若不爱你,拴住他干嘛!爱你的人,走到天涯也会回来找你。要费尽心机来挽留的,早晚有走丢的一天。
    """
    t7 = """
    我们都像小孩,胡闹是因为依赖你;懂礼貌,是因为你是外人。
    """
    
    
    def del_space(text):
        """
        去除无用的符号
        @param [in] test 待处理的字符串
        @retval 去除无用符号后的字符串
        """
        space_char = " `~!@#$%^&*()-_=+[{]}\\|\t;:'\",<.>/?……()——【】、;「」,。、"
        for c in space_char.decode('utf-8'):
            text = text.replace(c,'')
        return text
    
    def n_gram(text,n):
        """
        使用N_GRAM对全文进行分割
        @param [in] text 待分割的文本
        @param [in] n 1:uni-gram 2:bi-gram 3:tri-gram
        @return 分割后的字典列表[[词语,位置下标],......]
        """
        text = text.decode('utf-8')
        retlist = []
        for i in xrange(0,len(text)-n+1):
            short_text = ['',0]
            for j in xrange(0,n):
                short_text[0] += text[i+j]
            short_text[1] = i
            retlist.append(short_text)
        return retlist
    
    
    data = []       # 用于存储数据,将下标作为doc_id
    index = {}      # 用来记录词典及对应的倒排项
    def make_index(text):
        """
        制作倒排索引
        @param [in] text 待处理的文本
        """
        data.append(text)
        doc_id = len(data) - 1
        word_list = n_gram(del_space(text),2)
        for item in word_list:
            if not index.has_key(item[0]):
                index[item[0]] = set()
            index[item[0]].add(doc_id)
    
    
    def query(text):
        """
        查询
        @param [in] text 用于查询的短语或词语
        @return 匹配成功的文档下标doc_id
        """
        word_list = n_gram(del_space(text),2)
        doc_list = set()
        for item in word_list:
            if index.has_key(item[0]):
                if len(doc_list) == 0:
                    doc_list = index[item[0]]
                else:
                    doc_list = doc_list & index[item[0]]
        return doc_list
    
    
    if __name__ == "__main__":
        # 生成倒排索引
        make_index(t1)
        make_index(t2)
        make_index(t3)
        make_index(t4)
        make_index(t5)
        make_index(t6)
        make_index(t7)
        # 开始检索并遍历输出结果
        id_list = query("不要妄图拴住爱情")
        for item in id_list:
            print str(item)+':'+data[item]
    

    posted in GuaiK实验室 read more
  • guluting

    RabbitMQ

    安装

    curl -s -o rabbitmq.py https://raw.githubusercontent.com/guaik/GuaikInstaller/master/rabbitmq.py && python rabbitmq.py install

    卸载

    curl -s -o rabbitmq.py https://raw.githubusercontent.com/guaik/GuaikInstaller/master/rabbitmq.py && python rabbitmq.py remove


    MongoDB

    安装

    curl -s -o mongodb.py https://raw.githubusercontent.com/guaik/GuaikInstaller/master/mongodb.py && python mongodb.py install

    卸载

    curl -s -o mongodb.py https://raw.githubusercontent.com/guaik/GuaikInstaller/master/mongodb.py && python mongodb.py remove

    posted in GuaiK机房 read more
  • guluting

    其实当初想设计转发模块的时候是计划在框架内部实现负载均衡的功能的,但是经过测试发现效率并没有提升,我做了很多测试也尝试将代码在协程与多线程之间就行调优,但是速度始终不理想,最终还是使用Nginx去实现Tornado的多进程转发模式了。多进程配合Tornado的协程速度还是不错的。

    socket转发

    from tornado.tcpserver import TCPServer
    from tornado.ioloop  import IOLoop
    import tornado
    import socket
    import threading
    import logging
    from concurrent.futures import ThreadPoolExecutor
    import random
    
    from gevent import monkey
    monkey.patch_all()
    
    class Connection():
        def __init__(self,proxy, stream, address):
            print threading.current_thread()
            self._proxy = proxy
            self._proxy.clients.add(self)
            logging.debug("[connect] clients count:"+str(len(self._proxy.clients)))
            self._stream = stream
            self._address = address
            self._stream.set_close_callback(self.on_close)
            self.http_content = ""
            self.read_data()
    
        def read_data(self):
            self._stream.read_until("\r\n\r\n",self.analysis_head)
    
        def analysis_head(self, head):
            self.http_content = head
            fields = head.split("\r\n")
            field_map = {}
            for field in fields:
                kv = field.split(":")
                if len(kv) == 2:
                    field_map[kv[0].lower()] = kv[1]
            if field_map.has_key("content-length"):
                self._stream.read_bytes(int(field_map["content-length"]),self.proxy_data)
    
        def write_finish(self):
            if self._stream.closed():
                return
            self._stream.close()
    
        def recvall(self, sock):
            BUFF_SIZE = 1024
            data = b''
            while True:
                part = sock.recv(BUFF_SIZE)
                data += part
                if len(part) < BUFF_SIZE:
                    break
            return data
    
        def proxy_data(self, data):
            self.http_content += data
            logging.debug("http content:"+self.http_content)
            cur_server = ()
            if self._proxy.mode == 0:
                self._proxy.mutex.acquire()
                cur_server = self._proxy.server_list[self._proxy.cur_server]
                self._proxy.cur_server += 1
                if self._proxy.cur_server == self._proxy.server_count:
                    self._proxy.cur_server = 0
                self._proxy.mutex.release()
            if self._proxy.mode == 1:
                self._proxy.cur_server = random.randint(0,self._proxy.server_count-1)
                cur_server = self._proxy.server_list[self._proxy.cur_server]
    
            logging.debug("proxy -> "+cur_server[0]+":"+str(cur_server[1]))
    
            s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            s.connect(cur_server)
            s.send(self.http_content)
            ret_data = self.recvall(s)
            if not self._stream.closed():
                self._stream.write(ret_data,self.write_finish)
            s.close()
    
        def on_close(self):
            self._proxy.clients.remove(self)
            logging.debug("[close] clients count:"+str(len(self._proxy.clients)))
    
    
    class Proxy(TCPServer):
        def __init__(self, port, server_list, mode):
            super(Proxy,self).__init__()
            self.clients = set()
            self.port = port
            self.server_list = server_list
            self.mode = mode
            self.cur_server = 0
            self.server_count = len(server_list)
            self.mutex = threading.Lock()
            self.executor = ThreadPoolExecutor()
    
        def worker(self, stream, address):
            Connection(self,stream, address)
    
        def handle_stream(self, stream, address):
            # self.executor.submit(self.worker, stream, address)
            Connection(self,stream, address)
    
        def start(self):
            self.listen(self.port)
            IOLoop.instance().start()
    

    tornado.httpclient.AsyncHTTPClient()转发

    import tornado.ioloop
    import tornado.web
    import tornado.httpclient
    import tornado.gen
    import logging
    import threading
    
    from gevent import monkey
    monkey.patch_all()
    
    
    class MainHandler(tornado.web.RequestHandler):
    
        def get(self, *args, **kwargs):
            try:
                logging.debug(str(self.request.remote_ip)+"\n"+"-"*80+"\n"+str(self.request.body)+"\n"+"-"*80)
                self.application.analysis.decrypt_http(self.request.body,self)
            except Exception as e: logging.error(str(e))
    
        def handle_request(self, response):
            if not response.error:
                self.write(response.body)
            self.finish()
    
        @tornado.web.asynchronous
        def post(self, *args, **kwargs):
            try:
                logging.debug(str(self.request.remote_ip)+"\n"+"-"*80+"\n"+str(self.request.body)+"\n"+"-"*80)
                if self.application.mode == 0:
                    self.application.mutex.acquire()
                    cur_server = self.application.servers[self.application.cur_server]
                    self.application.cur_server += 1
                    if self.application.cur_server == self.application.server_count:
                        self.application.cur_server = 0
                    self.application.mutex.release()
                    logging.debug("proxy -> "+cur_server)
                    headers = self.request.headers
                    body = self.request.body
                    request = tornado.httpclient.HTTPRequest(cur_server,
                                                     method="POST", headers=headers, body=body, validate_cert=False)
                    tornado.httpclient.AsyncHTTPClient().fetch(request,self.handle_request)
            except Exception as e: logging.error(str(e))
    
    def make_app():
        return tornado.web.Application([
            (r"/", MainHandler),
        ])
    
    class Proxy():
        def __init__(self, port, servers, mode):
            self._port = port
            self._servers = servers
            self._mode = mode
    
        def start(self):
            app = make_app()
            app.listen(self._port)
            app.mode = self._mode
            app.servers = self._servers
            app.cur_server = 0
            app.server_count = len(self._servers)
            app.mutex = threading.Lock()
            tornado.ioloop.IOLoop.current().start()
    

    posted in GuaiK实验室 read more
  • guluting

    @pipi vscode还是比较好用的,我现在都很少用IDE了,除了开发Qt程序会用QtCreator😐

    posted in GuaiK重灾区 read more
  • guluting

    哈哈,其实我没看懂😀 可以介绍下作品吗

    posted in GuaiK设计师 read more