Group Details Private

GuaikSecurity

Information Security

Member List

  • CentOS7安装Kafka
    wget wget http://mirror.bit.edu.cn/apache/kafka/2.4.0/kafka_2.13-2.4.0.tgz
    tar -zxvf kafka_2.13-2.4.0.tgz
    cd kafka_2.13-2.4.0/bin
    # 运行zookeeper服务
    ./zookeeper-server-start.sh ../config/zookeeper.properties 1>/dev/null 2>&1 &
    # 运行kafka服务
    ./kafka-server-start.sh ../config/server.properties 1>/dev/null 2>&1 &
    

    安装KafkaManager

    yum install java-devel
    git clone https://github.com/yahoo/kafka-manager.git
    cd kafka-manager
    ./sbt clean dist
    cd /root/kafka-manager/target/universal/kafka-manager-x.x.x.x
    

    修改配置文件:vi conf/application.conf
    WX20200216-131314@2x.png

    运行KafkaManager

    cd bin
    ./kafka-manager -Dconfig.files=…/config/application.conf 1>/dev/null 2>&1 &
    

    常用指令:

    创建一个新的Topic:

     ./kafka-topics.sh --create --zookeeper hadoop1:2181,hadoop2:2181,hadoop3:2181 --replication-factor 1 --partitions 3 --topic TopicSession
    

    查询Topic列表:

    ./kafka-topics.sh --list --zookeeper hadoop1:2181,hadoop2:2181,hadoop3:2181
    

    查看Topic详细信息:

    ./kafka-topics.sh --describe --zookeeper hadoop1:2181,hadoop2:2181,hadoop3:2181 --topic TopicSession
    

    建立订阅者console-consumer:

    ./kafka-console-consumer.sh --bootstrap-server hadoop1:9092,hadoop2:9092,hadoop3:9092 --topic TopicSession
    

    建立发布者console-producer:

    ./kafka-console-producer.sh --broker-list hadoop1:9092,hadoop2:9092,hadoop3:9092 --topic TopicSession
    
    posted in GuaiK实验室
  • 20行代码爬取武汉2019-nCov最新信息
    # coding:utf-8
    from selenium import webdriver
    from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
    from selenium.webdriver.chrome.options import Options
    import json
    
    if __name__ == "__main__":
        chrome_options=Options()
        chrome_options.add_argument('--headless')
        browser = webdriver.Chrome(chrome_options=chrome_options)
        browser.get('https://m.baidu.com/s?word=%E6%AD%A6%E6%B1%89%E8%82%BA%E7%82%8E')
        browser.implicitly_wait(20)
        scripts = browser.find_elements_by_xpath('//script[@type="application/json"]')
        for item in scripts:
            j = json.loads(item.get_attribute('innerHTML'))
            if 'data' in j:
                if 'title' in j["data"] and j['data']['title'] == "事件脉络":
                    for es in j['data']['items']:
                        print(es['item_time_date']," ",es['eventDescription'])
        print(len(scripts))
        print(browser.title)
    

    数据来源:百度搜索【关键字:武汉肺炎】

    WX20200121-191448@2x.png

    posted in GuaiK实验室
  • PostgreSQL Array包含判断

    TABLE

    id p1
    1 888
    2 888
    3 888
    4 888
    5 888
    6 888
    select array[1,2,3] <@ (select array_agg(id) from mid where p1=888);
    // result: true
    select array[1,2,3,7] <@ (select array_agg(id) from mid where p1=888);
    // result: false
    select array[8,9,10] <@ (select array_agg(id) from mid where p1=888);
    // result: false
    select array[1,2] <@ (select array_agg(id) from mid where p1=888);
    // result: true
    

    ARRAY常用操作符

    Operator Description Example Result
    = equal ARRAY[1.1,2.1,3.1]::int[] = ARRAY[1,2,3] T
    <> not equal ARRAY[1,2,3] <> ARRAY[1,2,4] T
    < less than ARRAY[1,2,3] < ARRAY[1,2,4] T
    > greater than ARRAY[1,4,3] > ARRAY[1,2,4] T
    <= less than or equal ARRAY[1,2,3] <= ARRAY[1,2,3] T
    >= greater than or equal ARRAY[1,4,3] >= ARRAY[1,4,3] T
    @> contains ARRAY[1,4,3] @> ARRAY[3,1] T
    <@ is contained by ARRAY[2,7] <@ ARRAY[1,7,4,2,6] T
    && overlap (have elements in common) ARRAY[1,4,3] && ARRAY[2,1] T
    || array-to-array concatenation ARRAY[1,2,3] || ARRAY[4,5,6] {1,2,3,4,5,6}
    || array-to-array concatenation ARRAY[1,2,3] || ARRAY[[4,5,6],[7,8,9]] {{1,2,3},{4,5,6},{7,8,9}}
    || element-to-array concatenation 3 || ARRAY[4,5,6] {3,4,5,6}
    || array-to-element concatenation ARRAY[4,5,6] || 7 {4,5,6,7}
    posted in GuaiK实验室
  • dyld: malformed mach-o image: segment __DWARF has vmsize < filesize

    ☣️ 错误码:dyld: malformed mach-o image: segment __DWARF has vmsize < filesize

    🌀 现象:使用go run xxx.go不报错,使用go build xxx.go后运行./xxx会出现错误

    参数 版本
    OS macOS Catalina 10.15.2
    Glfw v3.3
    OpenGL v4.1-core

    ✅修复方式

    // -ldflags: 表示将后面的参数传给连接器
    // -s: 去掉符号信息
    // -w: 去掉DWARF调试信息
    go build -ldflags "-w" ./xxx.go
    
    posted in GuaiK实验室
  • Git修改最近一次commit的信息

    如果最近一次commit的信息填写错误需要修改,可以使用以下指令。

    git commit --amend
    
    posted in GuaiK实验室
  • PostgreSQL安装及使用UUID
    -- 安装uuid扩展函数
    create extension "uuid-ossp"
    
    -- 使用uuid生成
    select uuid_generate_v4() as uuid;
    

    WX20200115-123456.png

    posted in GuaiK实验室
  • RE: vue-js/element-ui

    我现在在使用Angular2 + Semantic UI,比较喜欢Semanntic UI的风格。😸

    posted in GuaiK实验室
  • RE: NodeBB头像上传后未发生改变

    最后我清理了下浏览器缓存,重新打开社区网站,发现头像已经被更新成功了。

    posted in GuaiK故障处理
  • NodeBB头像上传后未发生改变

    上传成功后点击“保存更改”,发现头像并没有变化,于是我开始从Nginx日志开始查,一直查到NodeBB的配置文件,都没有发现问题。👨‍💻 👨‍💻 👨‍💻

    posted in GuaiK故障处理
  • Python时间字符串转时间戳

    Python时间字符串转时间戳

    import time
    
    dt = '2020-01-01 00:00:00'
    ts = int(time.mktime(time.strptime(dt, "%Y-%m-%d %H:%M:%S")))
    print(ts)
    
    posted in GuaiK实验室