Initial commit - full huishou project

This commit is contained in:
jiapengyu
2026-07-27 14:07:26 +08:00
commit 60790ad1b3
39127 changed files with 5989265 additions and 0 deletions
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2019 Szymon Marczak
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+201
View File
@@ -0,0 +1,201 @@
# cacheable-lookup
> A cacheable [`dns.lookup(…)`](https://nodejs.org/api/dns.html#dns_dns_lookup_hostname_options_callback) that respects TTL :tada:
[![Build Status](https://travis-ci.org/szmarczak/cacheable-lookup.svg?branch=master)](https://travis-ci.org/szmarczak/cacheable-lookup)
[![Coverage Status](https://coveralls.io/repos/github/szmarczak/cacheable-lookup/badge.svg?branch=master)](https://coveralls.io/github/szmarczak/cacheable-lookup?branch=master)
[![npm](https://img.shields.io/npm/dm/cacheable-lookup.svg)](https://www.npmjs.com/package/cacheable-lookup)
[![install size](https://packagephobia.now.sh/badge?p=cacheable-lookup)](https://packagephobia.now.sh/result?p=cacheable-lookup)
Making lots of HTTP requests? You can save some time by caching DNS lookups :zap:
## Usage
### Using the `lookup` option
```js
const http = require('http');
const CacheableLookup = require('cacheable-lookup');
const cacheable = new CacheableLookup();
http.get('https://example.com', {lookup: cacheable.lookup}, response => {
// Handle the response here
});
```
### Attaching CacheableLookup to an Agent
```js
const http = require('http');
const CacheableLookup = require('cacheable-lookup');
const cacheable = new CacheableLookup();
cacheable.install(http.globalAgent);
http.get('https://example.com', response => {
// Handle the response here
});
```
## API
### new CacheableLookup(options)
Returns a new instance of `CacheableLookup`.
#### options
Type: `Object`<br>
Default: `{}`
Options used to cache the DNS lookups.
##### options.cacheAdapter
Type: [Keyv adapter instance](https://github.com/lukechilds/keyv)<br>
Default: `new Map()`
A Keyv adapter which stores the cache.
##### options.maxTtl
Type: `number`<br>
Default: `Infinity`
Limits the cache time (TTL in seconds).
If set to `0`, it will make a new DNS query each time.
##### options.resolver
Type: `Function`<br>
Default: [`new dns.Resolver()`](https://nodejs.org/api/dns.html#dns_class_dns_resolver)
An instance of [DNS Resolver](https://nodejs.org/api/dns.html#dns_class_dns_resolver) used to make DNS queries.
### Entry object
Type: `Object`
#### address
Type: `string`
The IP address (can be an IPv4 or IPv6 address).
#### family
Type: `number`
The IP family (`4` or `6`).
##### expires
Type: `number`
**Note**: This is not present when using the native `dns.lookup(...)`!
The timestamp (`Date.now() + ttl * 1000`) when the entry expires.
#### ttl
**Note**: This is not present when using the native `dns.lookup(...)`!
The time in seconds for its lifetime.
### Entry object (callback-style)
When `options.all` is `false`, then `callback(error, address, family, expires, ttl)` is called. <br>
When `options.all` is `true`, then `callback(error, entries)` is called.
### CacheableLookup instance
#### servers
Type: `Array`
DNS servers used to make the query. Can be overridden - then the new servers will be used.
#### [lookup(hostname, options, callback)](https://nodejs.org/api/dns.html#dns_dns_lookup_hostname_options_callback)
#### lookupAsync(hostname, options)
The asynchronous version of `dns.lookup(…)`.
Returns an [entry object](#entry-object).<br>
If `options.all` is true, returns an array of entry objects.
**Note**: If entry(ies) were not found, it will return `undefined`.
##### hostname
Type: `string`
##### options
Type: `Object`
The same as the [`dns.lookup(…)`](https://nodejs.org/api/dns.html#dns_dns_lookup_hostname_options_callback) options.
##### options.throwNotFound
Type: `boolean`<br>
Default: `false`
Throw when there's no match.
If set to `false` and it gets no match, it will return `undefined`.
**Note**: This option is meant **only** for the asynchronous implementation! The synchronous version will always give an error if no match found.
#### query(hostname, family)
An asynchronous function which returns cached DNS lookup entries. This is the base for `lookupAsync(hostname, options)` and `lookup(hostname, options, callback)`.
**Note**: This function has no options.
Returns an array of objects with `address`, `family`, `ttl` and `expires` properties.
#### queryAndCache(hostname, family)
An asynchronous function which makes a new DNS lookup query and updates the database. This is used by `query(hostname, family)` if no entry in the database is present.
Returns an array of objects with `address`, `family`, `ttl` and `expires` properties.
#### updateInterfaceInfo()
Updates interface info. For example, you need to run this when you plug or unplug your WiFi driver.
## High performance
See the benchmarks (queries `localhost`, performed on i7-7700k):
```
CacheableLookup#lookupAsync x 265,390 ops/sec ±0.65% (89 runs sampled)
CacheableLookup#lookupAsync.all x 119,187 ops/sec ±2.57% (87 runs sampled)
CacheableLookup#lookupAsync.all.ADDRCONFIG x 119,666 ops/sec ±0.75% (89 runs sampled)
CacheableLookup#lookup x 116,604 ops/sec ±0.68% (88 runs sampled)
CacheableLookup#lookup.all x 115,627 ops/sec ±0.72% (89 runs sampled)
CacheableLookup#lookup.all.ADDRCONFIG x 115,578 ops/sec ±0.90% (88 runs sampled)
CacheableLookup#lookupAsync - zero TTL x 60.83 ops/sec ±7.43% (51 runs sampled)
CacheableLookup#lookup - zero TTL x 49.22 ops/sec ±20.58% (49 runs sampled)
dns#resolve4 x 63.00 ops/sec ±5.88% (51 runs sampled)
dns#lookup x 21,303 ops/sec ±29.06% (35 runs sampled)
dns#lookup.all x 22,283 ops/sec ±21.96% (38 runs sampled)
dns#lookup.all.ADDRCONFIG x 5,922 ops/sec ±10.18% (38 runs sampled)
Fastest is CacheableLookup#lookupAsync
```
The package is based on [`dns.resolve4(…)`](https://nodejs.org/api/dns.html#dns_dns_resolve4_hostname_options_callback) and [`dns.resolve6(…)`](https://nodejs.org/api/dns.html#dns_dns_resolve6_hostname_options_callback).
[Why not `dns.lookup(…)`?](https://github.com/nodejs/node/issues/25560#issuecomment-455596215)
> It is not possible to use `dns.lookup(…)` because underlying calls like [getaddrinfo](http://man7.org/linux/man-pages/man3/getaddrinfo.3.html) have no concept of servers or TTL (caching is done on OS level instead).
## Related
- [cacheable-request](https://github.com/lukechilds/cacheable-request) - Wrap native HTTP requests with RFC compliant cache support
## License
MIT
+105
View File
@@ -0,0 +1,105 @@
import Keyv = require('keyv');
import {Resolver, LookupAddress} from 'dns';
import {Agent} from 'http';
type IPFamily = 4 | 6;
export interface Options {
/**
* A Keyv adapter which stores the cache.
* @default new Map()
*/
cacheAdapter?: Keyv;
/**
* Limits the cache time (TTL). If set to `0`, it will make a new DNS query each time.
* @default Infinity
*/
maxTtl?: number;
/**
* DNS Resolver used to make DNS queries.
* @default new dns.Resolver()
*/
resolver?: Resolver;
}
interface EntryObject {
/**
* The IP address (can be an IPv4 or IPv5 address).
*/
readonly address: string;
/**
* The IP family.
*/
readonly family: IPFamily;
/**
* The original TTL.
*/
readonly ttl: number;
/**
* The expiration timestamp.
*/
readonly expires: number;
}
interface LookupOptions {
/**
* One or more supported getaddrinfo flags. Multiple flags may be passed by bitwise ORing their values.
*/
hints?: number;
/**
* The record family. Must be `4` or `6`. IPv4 and IPv6 addresses are both returned by default.
*/
family?: IPFamily;
/**
* When `true`, the callback returns all resolved addresses in an array. Otherwise, returns a single address.
* @default false
*/
all?: boolean;
/**
* Throw when there's no match. If set to `false` and it gets no match, it will return `undefined`.
* @default false
*/
throwNotFound?: boolean;
}
export default class CacheableLookup {
constructor(options?: Options);
/**
* DNS servers used to make the query. Can be overridden - then the new servers will be used.
*/
servers: string[];
/**
* @see https://nodejs.org/api/dns.html#dns_dns_lookup_hostname_options_callback
*/
lookup(hostname: string, family: IPFamily, callback: (error: NodeJS.ErrnoException, address: string, family: IPFamily) => void): void;
lookup(hostname: string, callback: (error: NodeJS.ErrnoException, address: string, family: IPFamily) => void): void;
lookup(hostname: string, options: LookupOptions & {all: true}, callback: (error: NodeJS.ErrnoException, result: ReadonlyArray<EntryObject>) => void): void;
lookup(hostname: string, options: LookupOptions, callback: (error: NodeJS.ErrnoException, address: string, family: IPFamily) => void): void;
/**
* The asynchronous version of `dns.lookup(…)`.
*/
lookupAsync(hostname: string, options: LookupOptions & {all: true}): Promise<ReadonlyArray<EntryObject>>;
lookupAsync(hostname: string, options: LookupOptions): Promise<EntryObject>;
lookupAsync(hostname: string): Promise<EntryObject>;
lookupAsync(hostname: string, family: IPFamily): Promise<EntryObject>;
/**
* An asynchronous function which returns cached DNS lookup entries. This is the base for `lookupAsync(hostname, options)` and `lookup(hostname, options, callback)`.
*/
query(hostname: string, family: IPFamily): Promise<ReadonlyArray<EntryObject>>;
/**
* An asynchronous function which makes a new DNS lookup query and updates the database. This is used by `query(hostname, family)` if no entry in the database is present. Returns an array of objects with `address`, `family`, `ttl` and `expires` properties.
*/
queryAndCache(hostname: string, family: IPFamily): Promise<ReadonlyArray<EntryObject>>;
/**
* Attaches itself to an Agent instance.
*/
install(agent: Agent): void;
/**
* Removes itself from an Agent instance.
*/
uninstall(agent: Agent): void;
/**
* Updates interface info. For example, you need to run this when you plug or unplug your WiFi driver.
*/
updateInterfaceInfo(): void;
}
+216
View File
@@ -0,0 +1,216 @@
'use strict';
const {Resolver, V4MAPPED, ADDRCONFIG} = require('dns');
const {promisify} = require('util');
const os = require('os');
const Keyv = require('keyv');
const kCacheableLookupData = Symbol('cacheableLookupData');
const kCacheableLookupInstance = Symbol('cacheableLookupInstance');
const verifyAgent = agent => {
if (!(agent && typeof agent.createConnection === 'function')) {
throw new Error('Expected an Agent instance as the first argument');
}
};
const map4to6 = entries => {
for (const entry of entries) {
entry.address = `::ffff:${entry.address}`;
entry.family = 6;
}
};
const getIfaceInfo = () => {
let has4 = false;
let has6 = false;
for (const device of Object.values(os.networkInterfaces())) {
for (const iface of device) {
if (iface.internal) {
continue;
}
if (iface.family === 'IPv6') {
has6 = true;
} else {
has4 = true;
}
if (has4 && has6) {
break;
}
}
}
return {has4, has6};
};
class CacheableLookup {
constructor({cacheAdapter, maxTtl = Infinity, resolver} = {}) {
this.cache = new Keyv({
uri: typeof cacheAdapter === 'string' && cacheAdapter,
store: typeof cacheAdapter !== 'string' && cacheAdapter,
namespace: 'cached-lookup'
});
this.maxTtl = maxTtl;
this._resolver = resolver || new Resolver();
this._resolve4 = promisify(this._resolver.resolve4.bind(this._resolver));
this._resolve6 = promisify(this._resolver.resolve6.bind(this._resolver));
this._iface = getIfaceInfo();
this.lookup = this.lookup.bind(this);
this.lookupAsync = this.lookupAsync.bind(this);
}
set servers(servers) {
this._resolver.setServers(servers);
}
get servers() {
return this._resolver.getServers();
}
lookup(hostname, options, callback) {
if (typeof options === 'function') {
callback = options;
options = {};
}
// eslint-disable-next-line promise/prefer-await-to-then
this.lookupAsync(hostname, {...options, throwNotFound: true}).then(result => {
if (options.all) {
callback(null, result);
} else {
callback(null, result.address, result.family, result.expires, result.ttl);
}
}).catch(callback);
}
async lookupAsync(hostname, options = {}) {
let cached;
if (!options.family && options.all) {
const [cached4, cached6] = await Promise.all([this.lookupAsync(hostname, {all: true, family: 4}), this.lookupAsync(hostname, {all: true, family: 6})]);
cached = [...cached4, ...cached6];
} else {
cached = await this.query(hostname, options.family || 4);
if (cached.length === 0 && options.family === 6 && options.hints & V4MAPPED) {
cached = await this.query(hostname, 4);
map4to6(cached);
}
}
if (options.hints & ADDRCONFIG) {
const {_iface} = this;
cached = cached.filter(entry => entry.family === 6 ? _iface.has6 : _iface.has4);
}
if (cached.length === 0 && options.throwNotFound) {
const error = new Error(`ENOTFOUND ${hostname}`);
error.code = 'ENOTFOUND';
error.hostname = hostname;
throw error;
}
const now = Date.now();
cached = cached.filter(entry => entry.ttl === 0 || now < entry.expires);
if (options.all) {
return cached;
}
if (cached.length === 1) {
return cached[0];
}
if (cached.length === 0) {
return undefined;
}
return this._getEntry(cached);
}
async query(hostname, family) {
let cached = await this.cache.get(`${hostname}:${family}`);
if (!cached) {
cached = await this.queryAndCache(hostname, family);
}
return cached;
}
async queryAndCache(hostname, family) {
const resolve = family === 4 ? this._resolve4 : this._resolve6;
const entries = await resolve(hostname, {ttl: true});
if (entries === undefined) {
return [];
}
const now = Date.now();
let cacheTtl = 0;
for (const entry of entries) {
cacheTtl = Math.max(cacheTtl, entry.ttl);
entry.family = family;
entry.expires = now + (entry.ttl * 1000);
}
cacheTtl = Math.min(this.maxTtl, cacheTtl) * 1000;
if (this.maxTtl !== 0 && cacheTtl !== 0) {
await this.cache.set(`${hostname}:${family}`, entries, cacheTtl);
}
return entries;
}
_getEntry(entries) {
return entries[Math.floor(Math.random() * entries.length)];
}
install(agent) {
verifyAgent(agent);
if (kCacheableLookupData in agent) {
throw new Error('CacheableLookup has been already installed');
}
agent[kCacheableLookupData] = agent.createConnection;
agent[kCacheableLookupInstance] = this;
agent.createConnection = (options, callback) => {
if (!('lookup' in options)) {
options.lookup = this.lookup;
}
return agent[kCacheableLookupData](options, callback);
};
}
uninstall(agent) {
verifyAgent(agent);
if (agent[kCacheableLookupData]) {
if (agent[kCacheableLookupInstance] !== this) {
throw new Error('The agent is not owned by this CacheableLookup instance');
}
agent.createConnection = agent[kCacheableLookupData];
delete agent[kCacheableLookupData];
delete agent[kCacheableLookupInstance];
}
}
updateInterfaceInfo() {
this._iface = getIfaceInfo();
}
}
module.exports = CacheableLookup;
module.exports.default = CacheableLookup;
+45
View File
@@ -0,0 +1,45 @@
{
"name": "cacheable-lookup",
"version": "2.0.1",
"description": "A cacheable dns.lookup(…) that respects the TTL",
"engines": {
"node": ">=10"
},
"files": [
"index.js",
"index.d.ts"
],
"main": "index.js",
"scripts": {
"test": "xo && nyc ava && tsd"
},
"repository": {
"type": "git",
"url": "git+https://github.com/szmarczak/cacheable-lookup.git"
},
"keywords": [
"dns",
"lookup",
"cacheable",
"ttl"
],
"author": "Szymon Marczak",
"license": "MIT",
"bugs": {
"url": "https://github.com/szmarczak/cacheable-lookup/issues"
},
"homepage": "https://github.com/szmarczak/cacheable-lookup#readme",
"devDependencies": {
"ava": "^3.1.0",
"benchmark": "^2.1.4",
"coveralls": "^3.0.9",
"nyc": "^15.0.0",
"proxyquire": "^2.1.3",
"tsd": "^0.11.0",
"xo": "^0.25.3"
},
"dependencies": {
"@types/keyv": "^3.1.1",
"keyv": "^4.0.0"
}
}