yányào.com

Posts

poorman's ASR solution

Intro

大概20年h2期间,听闻feishu支持了会议自动转译字幕功能,but 当时私有化部署的版本一直没有支持,于是私下找了语音助手团队的同学企图蹭服务用于自动生成会议纪要,懒即是原动力

再 but 一下,出于服务成本考量没有蹭成功,后来对应的同学也陆续离职就没下文了

23年大模型开始各种火热起来,于是借助 openai 的 whisper 在本地陆续写了一些音频转文本的小任务,比如 yt-dlp 下载 bilibili 视频后转音频再转文本,个人习惯上阅读的效率还是高于视频形式的

另外结合语音识别和 LLM 的发散,也试着搓了点语音助手的尝试

但是实时语音识别场景,用 whiper-stream / sense-voice 这类方案还是会有识别不准确、内容丢失的问题存在,调整 step/length 参数效果也不明显,vad 方案延迟也很高…

New solution

上周忽然想起了 kaldi 项目也蛮经典了,于是就试着用 sherpa-[ncnn|onnx] 尝试了下 demo sherpa-[ncnn|onnx]-microphone 流式识别输出的整体效果已经很接近把音频文件喂给大模型的质量

于是联系维护者 @csukuangfj 请教怎么提升识别准确率,大佬过于卷周末还给造了新方案和输出优化,甚至还录了 b 站视频 https://www.bilibili.com/video/BV1xcEMz8EsX/

另外也体验到 two-pass 双模型识别的效果,两种方式都能满足俺的使用场景🎉

Pandoc generate pdf from markdown

Intro

始终不太习惯 word/wps 类工具编辑文本,再一次尝试用 pandoc 来生成 pdf,算是初步搞明白一点玩法,本文略过 markdown 的部分主要记录 latex 相关设定内容。

Setup

首先完整安装 latex 包

$ brew uninstall --cask basictex
$ brew install --cask mactex

Latex template

---
title: "Your article title here"
date: 2023-02-20
fontsize: 12pt
# fc-list :lang=zh
#mainfont: "PingFang SC"
mainfont: "Sarasa Mono SC"
# monofont: "FantasqueSansM Nerd Font Mono"
monofont: "Sarasa Mono SC"
# 中文换行支持
lang: zh-CN
geometry: margin=2cm

# 设置双栏
# classoption=twocolumn
---

在 markdown 文档的最前面加入以上 metadata 简化命令参数设定,实际效果等价于

Flair-58 Espresso Maker

flair-58

https://flairespresso.com/product/flair-58/

周末浪费了2天豆子,终于折腾出比较理想的设定

奶泡机没到位,先用娃的小奶锅煮 200ml Oatly 燕麦奶,稍热就好,没有打出泡沫也就无所谓拉花了
PlanB: 微波炉加热 + 宜家 ¥9.9 手持奶泡器

我买的优化后版本默认配无底手柄 + 58mm 18g粉碗,而上半年批次的好像是 20g粉碗,底部没有凹弧线,然后分水网和冲煮头是一体的,每杯都得拆下来清理,现在只需要拆手柄洗洗粉碗和分水网就好,方便连续出杯(但是对于没有多个手柄和粉碗的初阶用户来说,重新填粉加水的操作并不连续 -.-)
群里有大佬改装了底座和支架的方案,解决下压前倾问题和使用12cm+的秤

而无底手柄安装还不太熟练,经常要弯腰低头去看小翅膀有没有卡到位,然后注水再安装压力表(顺便当预浸泡),18g豆出 36g液体,如果粉磨得过细,下压会很费劲

虽然目前 5-9bar 之间压的曲线还不太稳定,偶尔通道也不太顺滋到杯外,但是香浓的 crema 已然远超预期(其实很容易过萃,但是家属喝不出来😂)

试了下用冰的屈臣氏做 coffee soda,难喝🤢

好多想买的配件:拉花奶缸、粉锤、布粉器、鼠尾粉碗、粉饼垃圾盒

update: 2022-08-19 14:09:08

买了新的拉花缸和称,但是懒到完全不想碰牛奶,only pure coffee and water…

早晨的冲煮开始逐渐暴躁,直接磨58g粉分3次装填、下压、出液到一个杯里,再加冰块分装成两份 long black 出门,少洗两次杯子动作也更流畅~大概15-20分钟完成

剁手好物推荐

Volvo v60

因为家庭新成员加入,所以买车的优先级提了上来。一开始考虑大尺寸6/7座suv,能装全家人短途出行,比如highlander, outlander, explorer, Pajero…后来因为今年各种新款上市,到最终提车时间有点晚,再加上城市拥堵和停车问题,回归到最初的选项也是一直种草的旅行车Volvo v60

中间还去看过Jaguar和Alfa Romeo,可惜后排空间有点小,对外型内饰非常满意,动力和操控也很可以,尤其Julie的前脸颜值太命中我了,一直对mx5这类笑脸无法抗拒…

V60在4s店试驾的豪华版,刚开始还挺不适应转向手感和油门刹车的,感觉都很沉很重。比起来以前开过的各种小钢炮和家用轿车,沉着有余灵活不够,前期不熟悉的时候操作起来都得小心翼翼。

最后订车选了致远焰影蓝运动款,因为厂家直销的关系也没讲价,就跟销售要了行车记录仪和儿童座椅。目前开两个月小一千公里的行程,大概总结些点吧

Self Host Maddy Mail Server

maddy

Composable all-in-one mail server. https://github.com/foxcpp/maddy

准备自建系列服务替换掉 google 全家桶,先试了传统的 Postfix, Dovecot, OpenDKIM, OpenSPF, OpenDMARC 套餐,但是本人水平菜,机器配置也不高,折腾两天还没跑起来,正好看到订阅的 changelog 推荐了 maddy 1 于是搓搓手气试试这个邮局方案,以下是满足个人喜好的优点

install

mkdir -p /data/maddy

# hard link caddy's crt and key to maddy
ln var/lib/caddy/.local/share/caddy/certificates/acme-v02.api.letsencrypt.org-directory/${domain}/${domain}.crt /data/maddy/${domain}.crt
ln var/lib/caddy/.local/share/caddy/certificates/acme-v02.api.letsencrypt.org-directory/${domain}/${domain}.key /data/maddy/${domain}.key


# foxcpp/maddy:v0.4.2
podman run \
  --name maddy \
  -v /data/maddy:/data \
  -p 25:25 \
  -p 143:143 \
  -p 587:587 \
  -p 993:993 \
  -p 465:465 \
  yanyaoer/maddy:master


# add user
podman run --rm -it -v /data/maddy:/data --entrypoint /bin/maddyctl yanyaoer/maddy:master creds create yanyao@mail.yadanhe.com

参考文档配置2 完后发送邮件对方可收,本地 thunderbird 接收回复邮件时有问题

Utteranc Comments

utterances - A lightweight comments widget built on GitHub issues. Use GitHub issues for blog comments, wiki pages and more!

install

  1. install utterances on your repo

  2. add the script to your template

<script src="https://utteranc.es/client.js"
        repo="[ENTER REPO HERE]"
        issue-term="pathname"
        theme="github-light"
        crossorigin="anonymous"
        async>
</script>

for exsamples https://github.com/yanyaoer/yanyaoer.github.io/commit/f2508ce7557d372f909f68ee32371f86895fd92f

done.

Zotero as personal knowledge management

Zotero 常用于科研的文献管理,由 firefox 浏览器插件发展到现在有了独立的桌面和移动客户端

quickstart

brew cask install zotero

这里简单用 caddy2 搭建 webdav 来同步附件,就没用高级的 zotfile

至于自带的笔记功能怎么说呢~ 已经适应不能这种风格了

add-onslink
quicklookhttps://github.com/mronkko/ZoteroQuickLook/releases
markdownhttps://addons.thunderbird.net/zh-cn/thunderbird/addon/markdown-here-xul/
papersgpthttps://github.com/papersgpt/papersgpt-for-zotero/releases
pdf2zhhttps://github.com/guaguastandup/zotero-pdf2zh/releases

subscription

将码农常用的文档工具 gitbook 转为 pdf

brew cask install calibre

sudo npm install -g gitbook-cli

gitbook install  # for gitbook plugin

git clone https://github.com/prometheus/docs.git

gitbook pdf

ps: gitbook 开源版本已经很久不维护了,可以换到 honkit fork 的版本

Rss2email

距离上次从 https://blogtrottr.com 迁移 rss 订阅到 feedly 差不多五六年了, 这会网络不可用的状况又变得严重些,干脆还是用自建服务来接收感兴趣的内容, 当然备选方案也不少:

出于复古心理顺便减少对应的 app 安装选择问题,又选择了回归 email 方式订阅更新

安装配置

rss2email 的安装到简单,参考 官方文档 直接 apt install rss2email,默认配置会发送文本邮件,修改 html-mail=True, 在我的环境里 smtp over sendgrid 很容易超配额,sendmail 基本发送不出去, 就配上 imap.google 账户自己发给自己啦~

从 feedly 导出 opml 文件然后 r2e opmlimport 完事大吉

需要注意的是,在 r2e run 的执行过程中对配置文件进行修改非常容易被覆盖 1,不要同时搞太多操作就好,

update: 2020-09-18 17:34:49

修改参数 force-from = True,默认会从 feed 里优先提取作者或者发布站点的邮箱,导致 gmail 过滤器失败

定时任务

现代 linux 发行版本(比如这里用的 ubuntu 20.04),默认不带 cronjob 服务,改为 systemd 来管理各种服务,定时任务的写法稍微麻烦了些,先用当前 user 权限来 $HOME/.config/systemd/user 写入定时任务配置

Deploy bitwarden_rs with podman

bitwarden_rs1 是个用 rust/rocket 编写的非官方 api 实现, dotnet 版本有点儿不太适合手头的部署环境

这里用比较现代的 podman2 来运行 docker 服务

podman run -d --name bitwarden -v /bw-data/:/data/:Z -e ROCKET_PORT=8080 -p 8080:8080 bitwardenrs/server:latest
podman generate systemd --name bitwarden --files
mv container-bitwarden.service /etc/systemd/system/

systemctl --user enable /etc/systemd/system/container-bitwarden.service
systemctl --user start container-bitwarden.service

然后用 nginx 或者 caddy 代理一下 8080 端口即可连接客户端

brew install bitwarden-cli  # npm install -g @bitwarden/cli
bw config server https://bw.myserver.com

update 2021-04-01 build bitwaden_rs from source

# install rust
apt install git make gcc libssl-dev pkg-config curl
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source $HOME/.cargo/env

# build bitwarden_rs
git clone https://github.com/dani-garcia/bitwarden_rs/
cd bitwarden_rs/
cargo build --features sqlite --release
cp target/release/bitwarden_rs /usr/local/bin/

# download bw_web_vault from https://github.com/dani-garcia/bw_web_builds/releases

edit /etc/systemd/system/bitwarden.service

Minio Storage Service

min.io 来自前 glusterFS 团队的分布式存储项目, 兼容 aws s3 / google cloud storage 接口,支持多磁盘、多节点,伸缩扩容方便, golang 编写+单执行文件部署,非常适合用 k8s 编排复制来搭建私有对象存储服务

没有生产环境的使用经验,以下仅用于业余尝鲜 六一节礼物

GNU/Linux 下载安装

wget https://dl.min.io/server/minio/release/linux-amd64/minio -O /usr/local/bin/minio
chmod +x /user/local/bin/minio

添加用户、组和配置文件

groupadd --system minio
useradd --system --gid minio --shell /usr/sbin/nologin --comment "Minio file server" minio
mkdir -p /data/minio
chown -R minio:minio /data/minio

# replace minio.service with your own config, eg. User,Group
wget https://raw.githubusercontent.com/minio/minio-service/master/linux-systemd/minio.service -O /etc/systemd/system/minio.service

配置端口和密钥

# optional. run `uuidgen` to creates AK/SK

cat <<EOF >> /tmp/minio
MINIO_VOLUMES="/data/minio/"
MINIO_OPTS="--address :9199"
MINIO_ACCESS_KEY=`uuidgen`
MINIO_SECRET_KEY=`uuidgen`

EOF

启动服务

systemctl enable minio.service
systemctl start minio.service

客户端 mc 使用说明参考官方文档好了,和 s3 命令行也没太多区别, 甚至可以用来做 gcs/s3 代理用

Pour Over Coffee Makers

办公室日常用品


每天一早 中午 到公司开始烧水,然后吭呲吭呲摇 40g 豆子兑 600ml 左右的水

手法就比较 随意 业余啦,主要参考下面两个视频,出品大概率自己都还挺喜欢的 按比例加注水量翻倍

“4:6 method” by Tetsu Kasuya 11

“The Ultimate V60 Technique” by James Hoffmann

家庭版


刚开始的时候装备上交了不少学费,比如:电动砍豆机廉价手磨aeropress, 虽然不能一概而论,实际上很大程度还是遵循一分钱一分货的原则,性能越好则价格呈指数上涨 条件允许的话尽量选择好一点的设备,毕竟也是天天要用的家伙事

Mastodon

Mastodon 长毛象1 – 基于 rubyonrails/reactjs/nodejs 开发的分布式 & 去中心化 twiter clone。利用空闲时间在 aws lightsail 上开了个实例把服务跑了起来

一开始走了些弯路,因为选机房和省钱的缘故,重建了若干次操作系统,最后的选择是 tokyo+cloudflare,没错我又套了 cdn,实在是海外线路到北京联通不稳定

安装步骤没有使用 docker 而是参考文档从源码安装2,原因和解决方案如下:

# create swapfile <https://linuxize.com/post/create-a-linux-swap-file/>
$ sudo fallocate -l 2G /swapfile
$ sudo chmod 600 /swapfile
$ sudo mkswap /swapfile
$ sudo swapon /swapfile

# verify active
$ sudo swapon --show
# optional: low value is better for production
$ sudo sysctl vm.swappiness=10

邮件配置

# By default the memory limit in Node. js is 512 mb, 
# use command —- max-old-space-size to increase the memory limit
$ NODE_OPTIONS="--max-old-space-size=4096" RAILS_ENV=production bundle exec rails assets:precompile

直接配置 localhost 发送邮件无效,先用 sendgrid 免费版顶着, 一天 100 封邮件目前够用

Gitlab Repo

gitlab 里面已经有十来个 group,几百个 project,虽然不一定会每个项目都参与, 但是为了快速同步代码,又捡起了 android 开发常用的 repo 命令来管理多项目代码。 这里使用了 gitlab graphql 接口来遍历项目生成分组的 manifest 文件

#!/usr/bin/env python

import argparse
import json
import os
import urllib2

GITLAB_HOST = "gitlab.mydomain.com"
GITLAB_SSH_URL = "ssh://git@" + GITLAB_HOST
GITLAB_GRAPHQL_URL = "https://" + GITLAB_HOST + "/api/graphql"
GITLAB_TOKEN = ""
GITLAB_GROUP = [""]
BLACK_LIST = ("",)

# 以上配置修改为自己的设定

template = """<?xml version="1.0" encoding="UTF-8"?>
<!-- autogen by gen.py, do not edit this file -->
<manifest>
  <remote  name="origin" fetch="{ssh_url}" />
  <default revision="master" remote="origin" sync-c="true" sync-j="4" />
{content}
</manifest>"""



def write_file(content, filename="default.xml"):
    _file = os.path.join(os.path.dirname(__file__), filename)
    with open(_file, "w") as f:
        f.write(template.format(content=content, ssh_url=GITLAB_SSH_URL))


def fetch_project_of_group(group, write=True):
    data = '{ group(fullPath: "%s") { projects { nodes { fullPath } } }}' % group
    req = urllib2.Request(
        GITLAB_GRAPHQL_URL,
        data=json.dumps(dict(query=data)),
        headers={
            "Authorization": "Bearer " + GITLAB_TOKEN,
            "Content-Type": "application/json",
        },
    )
    resp = json.load(urllib2.urlopen(req))
    content = "\n".join(
        [
            '<project path="{d}" name="{d}"/>'.format(d=p["fullPath"])
            for p in resp["data"]["group"]["projects"]["nodes"]
            if p["fullPath"] not in BLACK_LIST
        ]
    )

    if write:
        write_file(content, filename=group.replace("/", "-") + "-group.xml")
    else:
        return content


def main():
    parser = argparse.ArgumentParser(description="Process some projects.")
    parser.add_argument(
        "-G",
        "--group",
        help="for group/subgroup or all",
        type=str,
        choices=ns + ["all", "default"],
    )
    arg = parser.parse_args()

    if arg.group == "all":
        for x in GITLAB_GROUP:
            fetch_project_of_group(x)
        d = [fetch_project_of_group(x, write=False) for x in GITLAB_GROUP]
        write_file("\n".join(d))
    elif arg.group == "default":
        d = [fetch_project_of_group(x, write=False) for x in GITLAB_GROUP]
        write_file("\n".join(d))
    else:
        fetch_project_of_group(arg.group)


if __name__ == "__main__":
    main()

USAGE: 创建完 manifest 文件之后提交到仓库就可以愉快的玩耍啦

Dash Replacement with tmux

通过 tmux 快捷集成替换 dash.app 查询开发文档

$ brew install dasht

$ dasht-docsets | tr 'A-Z' 'a-z'
go
javascript
python_3
rust
tornado
# tmux.conf quick start
bind -n S-up command-prompt -p 'docset:' "splitw -h -fb -l 80 dasht '%%'"

Cloudflare Gost

从清明节开始,稳定运行好几年的 ss 服务器终于阵亡了,所有端口全挂。一直蹭公司的 vpn 查资料也挺到了五一,实在拖延够够的就再另外开了一台机器中转过去迁移数据,不过嘛年纪大了又开始犯懒,企图拯救获得资格认证的机器,通过一番网上冲浪学习到了目前(实测可用)能满足我需求的方案。简单来说就是:cloudflare[后文简称为 cf] + websockets over gost,实际的客户端通过 cdn 代理再接入服务

有几个需要注意的地方:

具体配置参考官方文档,一切浪费的时间都是源于没认真仔细看文档

update 2021-03-01 由于 shawdowsocks 的 android 客户端升级导致插件不可用,另外部署了 brook wsserver 给手机使用

webpack resolve local module

最近的项目刚开始,设计的目录层级有点深
经常会在好几层本地路径之间互相引用

import Image from '../../../../components/image'

这层层叠叠的路径写起来实在丑陋
不由得想起 Python 从项目根目录引用模块
然后研究了一下 Node.js 里的几种简易实现

干脆利落的软连接: ln -s node_modules src

修改环境变量: NODE_PATH=. node app

从本地目录安装:

// package.json 
// 需要运行 npm install
{
  "name": "baz",
  "dependencies": {
    "foo": "file: ./src",
  }
}

另外还有些修改 global,或者引入其他 require 实现的方法就不再一一列出了
最终选择的是修改 webpack 配置

// webpack.config.js
resolve: {
  modulesDirectories: [__dirname, 'node_modules'],
}

https://gist.github.com/branneman/8048520 http://stackoverflow.com/questions/10860244/how-to-make-the-require-in-node-js-to-be-always-relative-to-the-root-folder-of-t/41078266#41078266 https://webpack.github.io/docs/configuration.html#resolve-modulesdirectories

Leonard Cohen - you want it darker

年度最佳

you want it darker

you want it darker

循环听了好几天

干净低沉的嗓音加配乐

愈发担心以后听不到了怎么办 😢~

web audio

照文档撸了一下 AudioContext 可视化音频
桌面浏览器上 Safari 9, Chrome stable 绘制正常
移动端只有微信的 webview 能工作, 纯玩票叻 wave

ref: https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Visualizations_with_Web_Audio_API

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, user-scalable=no">
    <title>AV</title>
  </head>
  <body>
    <canvas id="vis"></canvas>
    <audio id="av" src="YOUR_AUDIO_FILE"></audio>
    <script src="index.js"></script>
  </body>
</html>
window.AudioContext = window.AudioContext || window.webkitAudioContext || window.mozAudioContext;

window.onload = function(){

  var canvas = document.getElementById('vis');
  var canvasCtx = canvas.getContext('2d');

  var isPlaying = true;
  var audio = document.getElementById('av');
  var audioCtx = new AudioContext();
  var analyser = audioCtx.createAnalyser();
  var audioSrc = audioCtx.createMediaElementSource(audio);
  audioSrc.connect(analyser);
  analyser.connect(audioCtx.destination);

  analyser.fftSize = 2048;
  var bufferLength = analyser.fftSize;
  dataArray = new Uint8Array(bufferLength);
  analyser.getByteTimeDomainData(dataArray);

  WIDTH = 500;
  HEIGHT = 150;
  function draw() {
    requestAnimationFrame(draw);
    if (!isPlaying) return;

    analyser.getByteTimeDomainData(dataArray);

    canvasCtx.fillStyle = 'rgb(200, 200, 200)';
    canvasCtx.fillRect(0, 0, WIDTH, HEIGHT);
    canvasCtx.lineWidth = 2;
    canvasCtx.strokeStyle = 'rgb(0, 0, 0)';
    canvasCtx.beginPath();

    var sliceWidth = WIDTH * 1.0 / bufferLength;
    var x = 0;

    for(var i = 0; i < bufferLength; i++) {

      var v = dataArray[i] / 128.0;
      var y = v * HEIGHT/2;

      if(i === 0) {
        canvasCtx.moveTo(x, y);
      } else {
        canvasCtx.lineTo(x, y);
      }

      x += sliceWidth;
    }

    canvasCtx.lineTo(canvas.width, canvas.height/2);
    canvasCtx.stroke();
  };

  draw();
  audio.play();

  document.body.onclick = function() {
    console.log(audio, isPlaying);
    if (isPlaying) {
      audio.pause();
      isPlaying = false;
    } else {
      audio.play();
      isPlaying = true;
    }
  }
}

move to caddy

去年用 Hexo 搭建的日志已经好几个月没更新了,最近休假有点空闲就继续更新吧。
先从 Github 迁移回自己的 Linode,然后安装一个 Caddyserver1 来渲染 markdown

Caddy is a unique web server with a modern feature set. Think nginx or Apache, but written in Go. With Caddy, you can serve your websites over HTTP/2. It can act as a reverse proxy and load balancer. Front your PHP apps with it. You can even deploy your site with git push. Cool, right?2

Download and install systemd

wget -O 'caddy.tar.gz' 'https://caddyserver.com/download/build?os=linux&arch=amd64&features=git%2Cipfilter'
tar zxf caddy.tar.gz
mkdir /opt/caddy
mkdir /var/run/caddy/
chown caddy:caddy -R /opt/caddy
chown caddy:caddy -R /var/run/caddy
mv caddy /opt/caddy/
useradd -d /var/run/caddy --system caddy

chmod 644 /etc/systemd/system/caddy.service
systemctl enable caddy.service
systemctl start caddy.service
systemctl status caddy.service

https://github.com/caddyserver/examples/blob/master/systemd%2Fcaddy.service

mongodb backup

有台机器准备2月份下架 记一个 mongodb 备份小脚本 :)

#!/bin/bash
# vim: set et sw=2 ts=2 sts=2 ff=unix fenc=utf8:

MONGO_DATABASE="_name_"

MONGO_HOST="_ip_"
MONGO_PORT="_prot_"
TIMESTAMP=`date +%Y-%m-%dT%H:%M:%S`
MONGODUMP_PATH="/usr/bin/mongodump"
BACKUPS_DIR="/data/dumps/"
BACKUP_NAME="$MONGO_DATABASE-$TIMESTAMP"

while test $# -gt 0
do
  case "$1" in
    -m) echo "backup mongthly and clear week_dir"
      #rm $BACKUPS_DIR"week/*"
      find $BACKUPS_DIR"week" -type f -name '*.tgz' -delete
      tar -czPf $BACKUPS_DIR"month/"$BACKUP_NAME.tgz $BACKUPS_DIR$MONGO_DATABASE
      ;;
    -w) echo "backup weekly"
      echo "tar -czPf $BACKUPS_DIR"week/"$BACKUP_NAME.tgz $BACKUPS_DIR$MONGO_DATABASE"
      ;;
    -d) echo "just dump"
      $MONGODUMP_PATH -d $MONGO_DATABASE --out $BACKUPS_DIR
      ;;
    *) echo "do nothing"
      ;;
  esac
  shift
done
# crontab -e
10 3 * * * /bin/bash $HOME/bin/mongobackup.sh -d > /dev/null 2>&1
20 3 * * 1 /bin/bash $HOME/bin/mongobackup.sh -w > /dev/null 2>&1
30 3 1 * * /bin/bash $HOME/bin/mongobackup.sh -m > /dev/null 2>&1

每天早上3点按天/周/月分别保存

let's encrypt

Let’s Encrypt 已经公开测试,不需要再提交测试域名表单,直接就能申请
小项目以后都能用这玩意开 https 不用花钱买证书哦啦啦

文档 做一遍给域名签上证书还挺简单的

ssl

# 获取项目代码
git clone https://github.com/letsencrypt/letsencrypt
cd letsencrypt

# 安装依赖
./letsencrypt-auto

# 获取证书
./letsencrypt-auto certonly --standalone -d www.example.com -d example.com
# 配置 nginx
server {
  listen 443 ;
  ssl on;
  ssl_certificate_key /etc/letsencrypt/live/youdomain/privkey.pem;
  ssl_certificate /etc/letsencrypt/live/youdomain/fullchain.pem;
}

需要注意的是 dnspod 等国内服务解析域名有问题
我这里直接切回 domains.google.com 就行了
使用 standalone 模式需要先停掉默认的 nginx
文档里提到可以使用 webroot 模式不用停 但我创建验证文件失败了 :(
默认90天过期,建议 crontab 定时更新

###Update

使用 acme.sh

或者 caddyserver

simple crawler

半夜看了本小说觉得翻页太累

首先要解决的问题是找一个质量还过得去的小说站 然后写个脚本去把它爬下来章节合并到一起

import fs from "fs";
import { argv } from "process";
import request from "request";
import cheerio from "cheerio";
import iconv from "iconv-lite";
import sanitize from 'sanitize-html';


class main {
  constructor() {
    Object.assign(this, {
      path: './chapter.json',
      html: './reader.html',
      url: {
        list: 'http://www.piaotian.net/html/6/6658/'
      },
      store: []
    })
  }
  fetch(url, callback) {
    request.get(url, {encoding: null}, (error, response, body)=> {
      if (!error && response.statusCode == 200) {
        let $ = cheerio.load(iconv.decode(body, 'GBK'));
        callback($, body);
      } else {
        console.log(url);
      }
    })
  }
  runchapter() {
    this.store.map(d=> {
      if (!d.content) {
        this.fetch(this.url.list + d.href, ($, body)=> {
          let content = iconv.decode(body, 'GBK');
          d.content = sanitize(content);
          this.save();
        })
      } else {
        console.log('runchapter fail: ', d.id, d.title);
      }
    })
    this.output();
  }
  output() {
    //let content = this.store.slice(0, 2)
    let content = this.store
      .map(d=> d.content)
      .join('<hr />');
    let html = `<html>
    <head>
      <meta charset="utf8" />
      <style>
        ul, table, div {
          display: none;
        }
        hr {
          height: 1px;
          margin: 4rem 0;
        }
        body {
          padding: 0 20%;
          font:24px/1.5 'Songti Sc';
        }
      </style>
    </head>
    <body>${content}</body></html>`;
    fs.writeFileSync(this.html, html, 'utf8');
  }
  reload() {
    this.fetch(this.url.list, ($)=> {
      this.store = [];
      $('.mainbody .centent ul li a').map((id, d)=> {
        let el = $(d);
        let href = el.attr('href');
        if (!href.endsWith('.html')) {
          return
        }
        this.store.push({ id, href, title: el.text() })
      })
      this.save();
      this.runchapter();
    })
  }
  config(force){
    if (force) {
      this.reload();
    } esle {
      let text = fs.readFileSync(this.path, 'utf8').toString();
      this.store = JSON.parse(text);
      this.runchapter();
    }
  }
  save() {
    fs.writeFileSync(this.path, JSON.stringify(this.store), 'utf8');
  }
  init() {
    let force = argv[2] === '-f';
    console.log('run: ', argv[2], force);
    this.config(force);
  }
}

new main().init();

于是就水出来一篇日志啦

send notification when task finish

终端里运行长时间任务(比如 make systemimage)的时候经常会切换到其他环境做别的事情 容易忘记查看之前的任务是否完成, 查到一些方法用在任务结束时发出通知

#C-z 切到后台运行
fg; tput bel

# Mac OS X
#系统弹窗
osascript -e 'tell app "System Events" to display alert "Build Completed" message "The checkout and build have completed."'

say "Job finished" #语音播报

#notification center
osascript -e 'display notification "Job finished" with title "Alert"'
sudo gem install terminal-notifier
terminal-notifier -message "Job finished!" -title "Alert"

# Ubuntu
notify-send "Job finished!"

# KDE
kdialog --passivepopup 'Job finished'

还有 iterm2 trigger 也能用来触发通知, 高亮文字

create CNAME with internationalized domain name

前段时间买了个 idn: yányào.com 闲置了很长时间没动 趁着十一长假无所事事的机会, 把玩了一下 hexo 挂到 yanyaoer.github.io

然后 CNAME 的时候掉坑了, 看到有人说30分钟生效傻傻等就不提了 实际上这个域名的 CNAME 内容应该用编码后的字符串而不是 yányào.com

https://github.com/yanyaoer/yanyaoer.github.io/commit/7c0e4e6863904442d368e3ad5c822f8f189bb7fc#diff-adc4bfdb0829dae99e3699393e3fbaa4

diff --git a/CNAME b/CNAME
index 6cb647c..92d166c 100644
--- a/CNAME
+++ b/CNAME
@@ -1 +1 @@
-yányào.com
+xn--ynyo-2nad.com

xtag-and-shadowdom

最近在做的项目重构, 原本打算用 reactjs, 写了一些实验代码后心累无爱 找了个 domdiff 配合自定义标签和 shadowdom 的独有作用域也是爽 YY

可惜的是测试红米上的 android webview 版本(32?)不支持 ::shadow 伪类 inline 方式覆盖样式略嫌繁琐

示例代码

xtag.js

let dom = {
  shadow(el) {
    return el.createShadowRoot ? el.createShadowRoot() : el.webkitCreateShadowRoot();
  },
  attr(el, prefix='') {
    return Object.keys(el.dataset).map((d)=> `${prefix}${d}="${el.dataset[d]}"`).join(' ')
  }
}

document.registerElement('x-image', {
  prototype: Object.create(HTMLElement.prototype, {
    createdCallback: {
      value() {
        //xtag 嵌套时这里读不到attr, 放到 attach
        console.log('onCreate::image');
      }
    },
    attachedCallback: {
      value() {
        console.log('onAttach::image');
        let shadow = dom.shadow(this);
        shadow.innerHTML = `<style>
            img { max-width: 100%; }
          </style>
          <img ${dom.attr(this)} />`;

        this.onclick = (e) => {
          console.log(this);
        }
      }
    }
  })
});

// other x-tag;

index.js

import 'babel/polyfill';
import './xtag.js';

class BaseView {
  constructor(opt) {
    Object.assign(this, {
      root: dom('#app')
    }, opt);
  }
  mount() {
    console.log('event::mount');
  }
  onCreate() {
    console.log('event::create');
  }
  onUpdate() {
    console.log('event::update');
  }
}

export default class BannerView extends BaseView {
  init(...args) {
    this.mount(...args);
  }
  render() {
    return `<x-image data-src="${this.src}"
                     data-href="${this.redirect_url}" />`;
  }
}

http://www.html5rocks.com/en/tutorials/webcomponents/customelements/ http://www.html5rocks.com/en/tutorials/webcomponents/shadowdom-301/

remote pbcopy with netcat

Quick start

while (true); do nc -l 2224 | pbcopy; done
#If your laptop is running linux, replacing pbcopy with xcopy should work:
#while (true); do nc -l 2224 | xcopy; done

echo "This text gets sent to clipboard" | nc localhost 2224

echo "RemoteForward 2224 localhost:2224" >> ~/.ssh/config
ssh remote -t 'cat blablabla | nc -q0 localhost 2224'

Daemonizing pbcopy

launchctl load ~/Library/LaunchAgents/local.pbcopy.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>localhost.pbcopy</string>
    <key>ProgramArguments</key>
    <array>
       <string>/usr/bin/pbcopy</string>
    </array>
    <key>inetdCompatibility</key>
    <dict>
       <key>Wait</key>
       <false/>
    </dict>
    <key>Sockets</key>
    <dict>
       <key>Listeners</key>
       <dict>
           <key>SockServiceName</key>
           <string>2224</string>
           <key>SockNodeName</key>
           <string>127.0.0.1</string>
       </dict>
    </dict>
</dict>
</plist>

Remote pbcopy

scp pbcopy remote:/path_to/pbcopy
chmod a+x path_to/pbcopy

#!/bin/bash

[ -n "$SSH_CLIENT" ] && SESSION_TYPE="remote"

if [[ $SESSION_TYPE == "remote" ]]; then
  cat | nc -q0 localhost 2224
else
  cat | pbcopy
fi

http://brettterpstra.com/2014/02/19/remote-pbcopy-on-os-x-systems/

forward email by postfix

sudo aptitude install postfix
hostname -f


# sudo vim /etc/postfix/main.cf
myhostname = example.com
myorigin = example.com
mydestination = example1.com, example2.com, ...
virtual_alias_maps = hash:/etc/postfix/virtual

# sudo vim /etc/postfix/virtual
@example1.com    name@forward.com
@example2.com    name@forward.com


sudo postmap /etc/postfix/virtual
sudo /etc/init.d/postfix reload

https://wiki.debian.org/Postfix#Forward_Emails

https://www.linode.com/docs/email/postfix/basic-postfix-email-gateway-on-debian-6-squeeze

https://www.debian-administration.org/article/243/Handling_mail_for_multiple_virtual_domains_with_postfix

mitmproxy

mitmproxy 是个命令行下查看/修改 http 请求的交互式工具

#截图 screenshoot screenshoot

#安装

sudo apt-get install python-dev libffi-dev
pip install mitmproxy

#使用

ubuntu 上启动 mitmproxy
mitmproxy –host

手机 设置 -> WLAN -> 代理
主机名: ubuntu 的 ip
端口: 8080

然后访问网络就会在 mitmproxy 里看到请求记录(如截图)

#快捷键

j,k 上下移动
enter 进入
tab 切换 request/response

#参考 http://mitmproxy.org/doc/mitmproxy.html
http://blog.philippheckel.com/2013/07/01/how-to-use-mitmproxy-to-read-and-modify-https-traffic-of-your-phone/

brick intro

#Introducing Brick: Minimal-markup Web Components for Faster App Development #介绍 brick: 用于快速开发 webapp 的自定义标签组件

https://hacks.mozilla.org/2013/08/introducing-brick-minimal-markup-web-components-for-faster-app-development/

Those of you on the cutting HTML5 edge may have already heard of the exciting Web Components specification. If you haven’t, you’ll probably want to read up on what makes this so exciting, but long story short, Web Components promise to open up a new realm of development by letting web developers write custom, reusable HTML tags. Think of them as JavaScript plugins without the need for additional code initialization or boilerplate markup/styling.

my osx setup

#setting

# Enable full keyboard access for all controls
defaults write NSGlobalDomain AppleKeyboardUIMode -int 3

# Disable menu bar transparency
defaults write NSGlobalDomain AppleEnableMenuBarTransparency -bool false

# Allow quitting Finder via ⌘ + Q; doing so will also hide desktop icons
defaults write com.apple.finder QuitMenuItem -bool true

# Avoid creating .DS_Store files on network volumes
defaults write com.apple.desktopservices DSDontWriteNetworkStores -bool true

# Disable the warning when changing a file extension
defaults write com.apple.finder FXEnableExtensionChangeWarning -bool false

# Enable tap to click (Trackpad)
defaults write com.apple.driver.AppleBluetoothMultitouch.trackpad Clicking -bool true

# Enable Safari’s debug menu
defaults write com.apple.Safari IncludeInternalDebugMenu -bool true

# To pin the dock to the right bottom
defaults write com.apple.dock pinning -string end
defaults write com.apple.dock orientation -string right

# Only Show Open Applications In The Dock
defaults write com.apple.dock static-only -bool true

defaults write com.apple.dock tilesize -int 24
defaults write com.apple.finder QLEnableTextSelection -bool true

# disable dashboard
defaults write com.apple.dashboard mcx-disabled -boolean true

#xcode with Command Line Tools