Need to build libuwsc example.c incorporating libev and libuwsc in single monolithic binary

I need to build example (from example.c) in partially static configuration, without libev.so and without libuwsc.so, which will not be available on a target system (that is, I will copy binary "example" to target system, but the target system will not have installed libev or libuwsc.)
Host (where I will build) is Ubuntu 18.04, 64-bit. Target (where I will execute) is Ubuntu 18.04, 64-bit, but has no shared libraries for libev and libuwsc.
I tested libuwsc initially as cloned from github - it works well (I tested example to communicate OK with existing websocket echo server.)
On host (where I will build), I tried changing cmake command from cmake .. to this:
cmake .. -DBUILD_STATIC_LIBS=ON -DBUILD_SHARED_LIBS=OFF
The result build process is now:
$ make
[ 11%] Building C object src/CMakeFiles/uwsc_s.dir/uwsc.c.o
[ 22%] Building C object src/CMakeFiles/uwsc_s.dir/log.c.o
[ 33%] Building C object src/CMakeFiles/uwsc_s.dir/utils.c.o
[ 44%] Building C object src/CMakeFiles/uwsc_s.dir/buffer/buffer.c.o
[ 55%] Building C object src/CMakeFiles/uwsc_s.dir/sha1.c.o
[ 66%] Building C object src/CMakeFiles/uwsc_s.dir/ssl.c.o
[ 77%] Linking C static library libuwsc.a
[ 77%] Built target uwsc_s
Scanning dependencies of target example
[ 88%] Building C object example/CMakeFiles/example.dir/example.c.o
[100%] Linking C executable example
[100%] Built target example

Now, example (binary) grows from 13408 to 42616 bytes, but when I move example binary to target system (target does not have libev or libuwsc intalled), executing ./example results:
./example: error while loading shared libraries: libev.so.4: cannot open shared object file: No such file or directory
The source system (where I am building) has libev.a (static) and libev.so (shared), but resulting binary example still is referencing libev.so, so it cannot start.
I am not expert on cmake, I think my problem is with with cmake.
Thank you for any help!

This problem turns out to be the way libev.a was generated when I retrieved it using apt.
I downloaded libev from the file repository area of
http://software.schmorp.de/pkg/libev.html
then did ./configure followed by make.
At this point in a directory .lib is the file libev.a, which is far larger than the same file installed using apt. This verison of libev.a was successfully linked using gcc command, and the resulting copy of example, while much larger (several hundred kilobytes), is indeed static as I wanted, and does run on a machine with no other installation of libev or libuwsc.
I am closing this issue as there is no remaining problem in this area.

I am closing this issue as there is no remaining problem in this area.


Router does not decode URIs

What version of Hono are you using?
4.3.6
What runtime/platform is your app running on?
Deno
What steps can reproduce the bug?
Deno/Hono
import { Hono } from 'npm:hono';

const h = new Hono();

h.get('/|', async (c) => c.text('ayy'));

Deno.serve({
	port: 8080,
}, h.fetch);

Node
// ayy
http.get({ host: 'localhost', port: 8080, path: '/|' }, res => res.pipe(process.stdout));'';

// 404 Not Found, expected ayy
http.get({ host: 'localhost', port: 8080, path: '/'+encodeURI('|') }, res => res.pipe(process.stdout));'';

What is the expected behavior?
ayy
What do you see instead?
404 Not Found
Additional information
No response

@yusukebe, do you think that it is a bug?

Hi @szmarczak
The router doesn't need to decode the path. Instead, if you want to decode it, you can use the getPath option for new Hono().
import { Hono } from 'hono'

const app = new Hono({
  getPath: (req) => {
    const url = req.url
    const queryIndex = url.indexOf('?', 8)
    const path = url.slice(url.indexOf('/', 8), queryIndex === -1 ? undefined : queryIndex)
    return decodeURI(path)
  },
})

I disagree. The params are decoded out-of-the-box (though in Request), I don't see why the path can't be:
Deno/Hono
import { Hono } from 'npm:hono';

const h = new Hono();

h.get('/:foobar', async (c) => c.text(c.req.param('foobar')));

Deno.serve({
	port: 8080,
}, h.fetch);

Node
http.get({ host: 'localhost', port: 8080, path: '/'+encodeURIComponent('/') }, res => res.pipe(process.stdout));'';

Also, the benchmark for find-my-way uses find which does decoding.

@szmarczak
Thanks. Indeed, paths may need to be decoded. But I think it should not be done by the router but by getPath(). We have made the method for that purpose. Regarding the benchmark, we can add a note on the page.
@usualoma What do you think about this?

should not be done by the router but by getPath()

There's no reason to make decoding opt-in. It's bad practice and bad DX.
Furthermore, the URI RFC says:

Percent-encoded octets must be decoded at some point during the
dereference process.  Applications must split the URI into its
components and subcomponents prior to decoding the octets, as
otherwise the decoded octets might be mistaken for delimiters.

so not decoding the components is actually against the spec.

I think the paths should be decoded by default, making decoding optional can lead to errors and poor developer experience (DX) because it introduces the possibility of inconsistent handling of URIs, I believe automatically decoding URI components ensures consistency and reduces the likelihood of bugs related to misinterpretation of encoded characters.

I think it should not be done by the router but by getPath()

I think we can add an option in Hono instance to disable decoding?
But default is enabled.

I considered it for a while and concluded that we should decode the path component before processing the routing.
As @yusukebe says, the place to do this processing is getPath(). We need an implementation of getPath() that is fast and does the decoding correctly.

@fzn0x @usualoma Thanks.
Let's implement getPath() to decode a URL path correctly and make it the default.

@yusukebe, can I implement it?

the place to do this processing is getPath()

Why? There have been several comments saying that with no rationale behind.

@szmarczak Thanks for your comment.
Are you saying that you think the router should do this process?
We believe that modifying getPath() would satisfy the specification. Please be specific about what you are objecting to.

@szmarczak
Oh, and to be clear, the "modifying getPath()" we are talking about now refers to the following changes.
We want to implement this change in a less performance-degrading way.
diff --git a/src/utils/url.ts b/src/utils/url.ts
index b37a5539..7109a307 100644
--- a/src/utils/url.ts
+++ b/src/utils/url.ts
@@ -73,7 +73,7 @@ export const getPath = (request: Request): string => {
   // Optimized: indexOf() + slice() is faster than RegExp
   const url = request.url
   const queryIndex = url.indexOf('?', 8)
-  return url.slice(url.indexOf('/', 8), queryIndex === -1 ? undefined : queryIndex)
+  return decodeURI(url.slice(url.indexOf('/', 8), queryIndex === -1 ? undefined : queryIndex))
 }

 export const getQueryStrings = (url: string): string => {

Are you saying that you think the router should do this process?

Correct. For example, testing this behavior would be much easier. Otherwise, you'd have to test the entire Hono app to make sure the behavior is right.

We believe that modifying getPath() would satisfy the specification.

Indeed, however the router can be used separately outside of Hono. Making it a user responsibility would be a bad DX.
For example, if I use a router then I'd expect it to parse paths for me without the need to do anything in that regards from my side.
If this is true when reading path params, then it should be true for the rest of the path.

Please be specific about what you are objecting to.

Having said the above, I don't believe that making it a part of getPath is a good idea.

We want to implement this change in a less performance-degrading way.

path.includes('%') ? decodeURI(path) : path

Also note that decodeURI may throw if it contains non-UTF-8 data so perhaps it's good to wrap it in a safe function that returns unwrapped data instead.

@szmarczak Thank you! I think I understand your thoughts. It will be helpful.
I consider the following
What are the responsibilities of Hono routers?
The match() method of Hono's router is responsible for taking a string and returning a result.
https://github.com/honojs/hono/blob/main/src/router.ts#L7-L11
If it is "receive a Request object and return a result", I think it should be decoded at the router, but since it is not, I feel comfortable with the API being "decode and pass it outside the router".
Who do we consider DX for?

the router can be used separately outside of Hono

Yes, there may be such users.
But I would prefer to improve the DX for the users of Hono (or us, the developers) who implement their own router in Hono, rather than improve the DX for the users who "use the router outside of Hono". If I want to write a new router, I would appreciate it if it is decoded outside the router.
Performance
path.includes('%') ? decodeURI(path) : path

I think that snippet will meet the specification, but we need code that performs better. For example, the following code. (Whether it is decodeURI or decodeURIComponent is a separate discussion.)
export const getPath = (request: Request): string => {
  const url = request.url
  const start = url.indexOf('/', 8)
  let i = start
  for (; i < url.length; i++) {
    const charCode = url.charCodeAt(i)
    if (charCode === 37) {
      // '%'
      // If the path contains percent encoding, use `indexOf()` to find '?' and return the result immediately.
      // Although this is a performance disadvantage, it is acceptable since we prefer cases that do not include percent encoding.
      const queryIndex = url.indexOf('?', i)
      return decodeURIComponent(url.slice(start, queryIndex === -1 ? undefined : queryIndex))
    } else if (charCode === 63) {
      // '?'
      break
    }
  }
  return url.slice(start, i)
}

The match() method of Hono's router is responsible for taking a string

The string consists of URI components.

If it is "receive a Request object and return a result", I think it should be decoded at the router, but since it is not

Whether the URL is extracted from Request or a user supplies its own request.url directly (guaranteed it's not decoded already) is irrelevant.

If I want to write a new router, I would appreciate it if it is decoded outside the router.

Then you can extract the custom decoding URI function into fastDecodeURI and do path = fastDecodeURI(path) in
https://github.com/honojs/hono/blob/d87d9964433a4777b09abff9360ff12643d00440/src/router/trie-router/router.ts#L25
and other routers too. IMO in that way DX for Hono developers is preserved.

I think that snippet will meet the specification, but we need code that performs better. For example, the following code.

I haven't benchmarked this but it may occur that for + charCodeAt is slower than calling always indexOf('?') and includes('%'). (Well, after all it's JS and not C++ so often the code that "looks fast" is not). Feel free to check that, I'll be back in the morning as I'm off to sleep.

I think that both sides (me and Hono) have expressed their opinions thoroughly. Whether it's getPath or router is a just implementation detail of free choice. It doesn't matter for the Hono end user which way it's done. I don't think I have any other points to add. Thanks for consideration!

Regarding the benchmark, we can add a note on the page.

I think the benchmark should contain code that decodes the URI (using Hono's implementation) for routers that don't decode URIs. Otherwise the benchmark is untrue as it doesn't represent how the route is handled in a real life scenario.

@szmarczak Thanks. I understand that there are other opinions, but I still think that getPath() should be modified.

I would like to refer to #2688 for discussion.
If anyone has an opinion, please comment.
Expected Routing Results
Is this the expected routing result?
https://github.com/honojs/hono/pull/2688/files#diff-8ac13809c9886e994d1db33943de82df4d6c5d88b73fd0270c0189804ff565c2R736-R775
There may be disagreement as to whether http://localhost/users/hono%2Fposts should match /users/:id' or /users/:id/posts'. The current v4.3.7, before the change, matches /users/:id, so this is a spec change.
It's a bit of a quibble whether http://localhost/users/%73tatic should match /users/static, but it might slip through and match '/users/:id' and c.req.param ('id') returns "static", then there may be a security problem depending on the application specification, so I think it should match /users/static. This is also a spec change.
Performace
I have benchmarked with benchmarks/utils/src/get-path.ts and I think this implementation will be faster in any runtime.
% bun run src/get-path.ts; deno run --allow-sys src/get-path.ts; npx tsx src/get-path.ts
cpu: Apple M2 Pro
runtime: bun 1.1.2 (arm64-darwin)

benchmark                                time (avg)             (min … max)       p75       p99      p999
--------------------------------------------------------------------------- -----------------------------
noop                                     45 ps/iter       (0 ps … 22.16 ns)     41 ps     81 ps    122 ps !

• getPath
--------------------------------------------------------------------------- -----------------------------
slice + indexOf : w/o decodeURI       30.43 ns/iter      (28.4 ns … 307 ns)  29.97 ns  36.03 ns    212 ns
regexp : w/o decodeURI                40.58 ns/iter     (34.79 ns … 312 ns)  39.67 ns  53.65 ns    285 ns
slice + indexOf                       63.21 ns/iter     (56.68 ns … 343 ns)  63.99 ns  72.98 ns    280 ns
slice + for-loop + flag               23.35 ns/iter     (21.81 ns … 313 ns)  22.66 ns   29.3 ns    211 ns
slice + for-loop + immediate return   23.12 ns/iter     (21.67 ns … 235 ns)  22.66 ns   27.3 ns    188 ns

summary for getPath
  slice + for-loop + immediate return
   1.01x faster than slice + for-loop + flag
   1.32x faster than slice + indexOf : w/o decodeURI
   1.75x faster than regexp : w/o decodeURI
   2.73x faster than slice + indexOf
cpu: Apple M2 Pro
runtime: deno 1.43.1 (aarch64-apple-darwin)

benchmark                                time (avg)             (min … max)       p75       p99      p999
--------------------------------------------------------------------------- -----------------------------
noop                                    698 ps/iter         (0 ps … 977 ns)      0 ps      0 ps      0 ps !

• getPath
--------------------------------------------------------------------------- -----------------------------
slice + indexOf : w/o decodeURI       15.12 ns/iter         (0 ps … 977 ns)      0 ps    977 ns    977 ns !
regexp : w/o decodeURI                49.17 ns/iter         (0 ps … 977 ns)      0 ps    977 ns    977 ns !
slice + indexOf                       45.34 ns/iter         (0 ps … 977 ns)      0 ps    977 ns    977 ns !
slice + for-loop + flag                34.2 ns/iter         (0 ps … 977 ns)      0 ps    977 ns    977 ns !
slice + for-loop + immediate return   31.73 ns/iter         (0 ps … 977 ns)      0 ps    977 ns    977 ns !

summary for getPath
  slice + indexOf : w/o decodeURI
   2.1x faster than slice + for-loop + immediate return
   2.26x faster than slice + for-loop + flag
   3x faster than slice + indexOf
   3.25x faster than regexp : w/o decodeURI
cpu: Apple M2 Pro
runtime: node v20.0.0 (arm64-darwin)

benchmark                                time (avg)             (min … max)       p75       p99      p999
--------------------------------------------------------------------------- -----------------------------
noop                                     71 ps/iter        (20 ps … 234 ns)     82 ps    102 ps    143 ps !

• getPath
--------------------------------------------------------------------------- -----------------------------
slice + indexOf : w/o decodeURI       21.29 ns/iter     (19.96 ns … 177 ns)  21.18 ns  30.03 ns  40.85 ns
regexp : w/o decodeURI                42.16 ns/iter     (37.82 ns … 259 ns)  40.85 ns    105 ns    143 ns
slice + indexOf                       34.05 ns/iter     (31.74 ns … 146 ns)     34 ns  42.07 ns    125 ns
slice + for-loop + flag                27.9 ns/iter     (26.12 ns … 304 ns)  27.73 ns  37.03 ns    140 ns
slice + for-loop + immediate return   27.39 ns/iter     (25.45 ns … 192 ns)  27.02 ns  36.44 ns    153 ns

summary for getPath
  slice + indexOf : w/o decodeURI
   1.29x faster than slice + for-loop + immediate return
   1.31x faster than slice + for-loop + flag
   1.6x faster than slice + indexOf
   1.98x faster than regexp : w/o decodeURI

Handle URIError: URI malformed
Error handling is under consideration.

@usualoma decodeURIComponent is incorrect. decodeURI is correct. It must not normalize %2F into / (nor %3F otherwise it will mistake it for a query).
localhost/foo%2Fbar/baz/ąę consists of only one segment, not two. If the router accepted an array, it would look like this:
['foo/bar', 'baz', 'ąę']

However, if we're using strings, then to match this we need foo%2Fbar/baz/ąę (note that ą and ę are decoded and not in their percent-encoded form).

@szmarczak Thanks for the comment!

(nor %3F otherwise it will mistake it for a query).

If you read the code, you will see that we extract the path component and then apply decodeURIComponent, so there is no confusion with the query.

localhost/foo%2Fbar/baz/ąę consists of only three segment

Can you provide a reason to believe that the specification is correct? I am referring to RFC 3986, 2.3, which states that they should be decoded.
https://datatracker.ietf.org/doc/html/rfc3986#section-2.3

For consistency, percent-encoded octets in the ranges of ALPHA
(%41-%5A and %61-%7A), DIGIT (%30-%39), hyphen (%2D), period (%2E),
underscore (%5F), or tilde (%7E) should not be created by URI
producers and, when found in a URI, should be decoded to their
corresponding unreserved characters by URI normalizers.

Also, /a%2Fb.html is considered the same as /a/b.html on typical web servers such as nginx.

Can you provide a reason to believe that the specification is correct?

I don't understand the question. The specification is always correct because it is the definition.

I am referring to RFC 3986, 2.3, which states that they should be decoded.

2F is not in the range of RFC 3986, 2.3 so you're incorrectly adhering to it.

Also, /a%2Fb.html is considered the same as /a/b.html on typical web servers such as nginx.

We're not talking about nginx or any other servers here.

https://www.rfc-editor.org/rfc/rfc9110.html#name-https-normalization-and-com

Characters other than those in the "reserved" set are equivalent to their percent-encoded octets: the normal form is to not encode them (see Sections 2.1 and 2.2 of [URI]).

https://www.rfc-editor.org/rfc/rfc3986#section-2.2

2.2.  Reserved Characters
  reserved    = gen-delims / sub-delims

  gen-delims  = ":" / "/" / "?" / "#" / "[" / "]" / "@"

  sub-delims  = "!" / "$" / "&" / "'" / "(" / ")"
              / "*" / "+" / "," / ";" / "="

I hope this clarifies. This means you must not decode %3F into ? while normalizing the path segment (normalization happens when we operate on paths like strings, if segments were arrays then decodeURIComponent would be correct, like said in the comment above).

@szmarczak
Sorry, you were right from the beginning on the following points you mentioned and my understanding was wrong. If we look at it in terms of "segments", I think it should be divided into three.

localhost/foo%2Fbar/baz/ąę consists of only three segments, not four. If the router accepted an array, it would look like this: > localhost/foo%2Fbar/baz/ąę
['foo/bar', 'baz', 'ąę']

However, "apply decodeURI() -> split into segments -> apply decodeURIComponent() for each segment" doesn't work.
decodeURI("%2525a/b/c").split("/").map(decodeURIComponent)

The expected result should be as follows,
['%25a', 'b', 'c']

In fact, it would look like this.
['%a', 'b', 'c']

If we want to get the correct result on a per-segment basis, we would need to return an array from getPath() as follows
return "%2525a/b/c".split("/").map(decodeURIComponent)

However, the Hono router does not understand the unit of “segment”.

No worries! I'm glad I could clarify.

However, "apply decodeURI() -> split into segments -> apply decodeURIComponent() for each segment" doesn't work.

I never said it would :) It just depends whether the path is a string or an array of strings. In this case, Hono's path is just a string therefore decodeURI should be used as mentioned in this comment. This is as simple as replacing decodeURIComponent with decodeURI in your PR.
In array notation, ['foo/bar', 'baz', 'ąę'] means foo%2Fbar/baz/ąę as a decoded (normalized) string.
hono.get('/foo%2Fbar/baz/ąę', ...)

I believe implement memoization could improve the speed. Also skips normal path to fast parse

Click me
```js
/**
 * Parse the `req` URL with memoization.
 *
 * @param {ServerRequest} req
 * @return {Object|undefined}
 * @public
 */
function parseurl(req) {
  const url = req.url;
if (url === undefined) {
// URL is undefined
return undefined;
}
const parsed = req._parsedUrl;
if (isFresh(url, parsed)) {
// Return cached URL parse
return parsed;
}
// Parse the URL
const newParsed = fastparse(url);
newParsed._raw = url;
return (req._parsedUrl = newParsed);
}
/**

Parse the req original URL with fallback and memoization.

@param {ServerRequest} req
@return {Object|undefined}
@public
*/
function originalurl(req) {
const url = req.originalUrl;

if (typeof url !== 'string') {
// Fallback
return parseurl(req);
}
const parsed = req._parsedOriginalUrl;
if (isFresh(url, parsed)) {
// Return cached URL parse
return parsed;
}
// Parse the URL
const newParsed = fastparse(url);
newParsed._raw = url;
return (req._parsedOriginalUrl = newParsed);
}
/**

Parse the str URL with a fast-path shortcut.

@param {string} str
@return {Object}
@private
/
function fastparse(str) {
if (typeof str !== 'string' || str.charCodeAt(0) !== 0x2f / / */) {
return decodeURIToUrlObject(str);
}

let pathname = str;
let query = null;
let search = null;
// Unroll the regexp for performance optimization
for (let i = 1; i < str.length; i++) {
switch (str.charCodeAt(i)) {
case 0x3f: /* ? /
if (search === null) {
pathname = str.substring(0, i);
query = str.substring(i + 1);
search = str.substring(i);
}
break;
case 0x09: / \t /
case 0x0a: / \n /
case 0x0c: / \f /
case 0x0d: / \r /
case 0x20: / /
case 0x23: / # /
case 0xa0: / non-breaking space /
case 0xfeff: / BOM */
return decodeURIToUrlObject(str);
}
}
const url = {};
url.path = str;
url.href = str;
url.pathname = pathname;
if (search !== null) {
url.query = query;
url.search = search;
}
return url;
}
/**

Decode the URI and convert it to a URL object.

@param {string} uri
@return {Object}
@private
*/
function decodeURIToUrlObject(uri) {
const decodedURI = decodeURI(uri);
const url = new URL(decodedURI); // Base URL to ensure correct parsing

return {
href: decodedURI,
path: url.pathname + (url.search || ''),
pathname: url.pathname,
search: url.search,
query: url.searchParams.toString(),
};
}
/**

Determine if parsed URL is still fresh.

@param {string} url
@param {object} parsedUrl
@return {boolean}
@private
*/
function isFresh(url, parsedUrl) {
return (
typeof parsedUrl === 'object' &&
parsedUrl !== null &&
parsedUrl._raw === url
);
}

const req = {
url: "localhost:8000/foo%2Fbar/baz/ąę"
}
console.log(parseurl(req).path)

Result

```js

/**
 * Parse the `req` URL with memoization.
 *
 * @param {ServerRequest} req
 * @return {Object|undefined}
 * @public
 */
function parseurl(req) {
  const url = req.url;

  if (url === undefined) {
    // URL is undefined
    return undefined;
  }

  const parsed = req._parsedUrl;

  if (isFresh(url, parsed)) {
    // Return cached URL parse
    return parsed;
  }

  // Parse the URL
  const newParsed = fastparse(url);
  newParsed._raw = url;

  return (req._parsedUrl = newParsed);
}

/**
 * Parse the `req` original URL with fallback and memoization.
 *
 * @param {ServerRequest} req
 * @return {Object|undefined}
 * @public
 */
function originalurl(req) {
  const url = req.originalUrl;

  if (typeof url !== 'string') {
    // Fallback
    return parseurl(req);
  }

  const parsed = req._parsedOriginalUrl;

  if (isFresh(url, parsed)) {
    // Return cached URL parse
    return parsed;
  }

  // Parse the URL
  const newParsed = fastparse(url);
  newParsed._raw = url;

  return (req._parsedOriginalUrl = newParsed);
}

/**
 * Parse the `str` URL with a fast-path shortcut.
 *
 * @param {string} str
 * @return {Object}
 * @private
 */
function fastparse(str) {
  if (typeof str !== 'string' || str.charCodeAt(0) !== 0x2f /* / */) {
    return decodeURIToUrlObject(str);
  }

  let pathname = str;
  let query = null;
  let search = null;

  // Unroll the regexp for performance optimization
  for (let i = 1; i < str.length; i++) {
    switch (str.charCodeAt(i)) {
      case 0x3f: /* ? */
        if (search === null) {
          pathname = str.substring(0, i);
          query = str.substring(i + 1);
          search = str.substring(i);
        }
        break;
      case 0x09: /* \t */
      case 0x0a: /* \n */
      case 0x0c: /* \f */
      case 0x0d: /* \r */
      case 0x20: /*   */
      case 0x23: /* # */
      case 0xa0: /* non-breaking space */
      case 0xfeff: /* BOM */
        return decodeURIToUrlObject(str);
    }
  }

  const url = {};
  url.path = str;
  url.href = str;
  url.pathname = pathname;

  if (search !== null) {
    url.query = query;
    url.search = search;
  }

  return url;
}

/**
 * Decode the URI and convert it to a URL object.
 *
 * @param {string} uri
 * @return {Object}
 * @private
 */
function decodeURIToUrlObject(uri) {
  const decodedURI = decodeURI(uri);
  const url = new URL(decodedURI); // Base URL to ensure correct parsing

  return {
    href: decodedURI,
    path: url.pathname + (url.search || ''),
    pathname: url.pathname,
    search: url.search,
    query: url.searchParams.toString(),
  };
}

/**
 * Determine if parsed URL is still fresh.
 *
 * @param {string} url
 * @param {object} parsedUrl
 * @return {boolean}
 * @private
 */
function isFresh(url, parsedUrl) {
  return (
    typeof parsedUrl === 'object' &&
    parsedUrl !== null &&
    parsedUrl._raw === url
  );
}

const req = {
  url: "localhost:8000/foo%2Fbar/baz/ąę"
}
console.log(parseurl(req).path)

I believe implement memoization could improve the speed. Also skips normal path to fast parse

Click me
/**
 * Parse the `req` URL with memoization.
 *
 * @param {ServerRequest} req
 * @return {Object|undefined}
 * @public
 */
function parseurl(req) {
  const url = req.url;

  if (url === undefined) {
    // URL is undefined
    return undefined;
  }

  const parsed = req._parsedUrl;

  if (isFresh(url, parsed)) {
    // Return cached URL parse
    return parsed;
  }

  // Parse the URL
  const newParsed = fastparse(url);
  newParsed._raw = url;

  return (req._parsedUrl = newParsed);
}

/**
 * Parse the `req` original URL with fallback and memoization.
 *
 * @param {ServerRequest} req
 * @return {Object|undefined}
 * @public
 */
function originalurl(req) {
  const url = req.originalUrl;

  if (typeof url !== 'string') {
    // Fallback
    return parseurl(req);
  }

  const parsed = req._parsedOriginalUrl;

  if (isFresh(url, parsed)) {
    // Return cached URL parse
    return parsed;
  }

  // Parse the URL
  const newParsed = fastparse(url);
  newParsed._raw = url;

  return (req._parsedOriginalUrl = newParsed);
}

/**
 * Parse the `str` URL with a fast-path shortcut.
 *
 * @param {string} str
 * @return {Object}
 * @private
 */
function fastparse(str) {
  if (typeof str !== 'string' || str.charCodeAt(0) !== 0x2f /* / */) {
    return decodeURIToUrlObject(str);
  }

  let pathname = str;
  let query = null;
  let search = null;

  // Unroll the regexp for performance optimization
  for (let i = 1; i < str.length; i++) {
    switch (str.charCodeAt(i)) {
      case 0x3f: /* ? */
        if (search === null) {
          pathname = str.substring(0, i);
          query = str.substring(i + 1);
          search = str.substring(i);
        }
        break;
      case 0x09: /* \t */
      case 0x0a: /* \n */
      case 0x0c: /* \f */
      case 0x0d: /* \r */
      case 0x20: /*   */
      case 0x23: /* # */
      case 0xa0: /* non-breaking space */
      case 0xfeff: /* BOM */
        return decodeURIToUrlObject(str);
    }
  }

  const url = {};
  url.path = str;
  url.href = str;
  url.pathname = pathname;

  if (search !== null) {
    url.query = query;
    url.search = search;
  }

  return url;
}

/**
 * Decode the URI and convert it to a URL object.
 *
 * @param {string} uri
 * @return {Object}
 * @private
 */
function decodeURIToUrlObject(uri) {
  const decodedURI = decodeURI(uri);
  const url = new URL(decodedURI); // Base URL to ensure correct parsing

  return {
    href: decodedURI,
    path: url.pathname + (url.search || ''),
    pathname: url.pathname,
    search: url.search,
    query: url.searchParams.toString(),
  };
}

/**
 * Determine if parsed URL is still fresh.
 *
 * @param {string} url
 * @param {object} parsedUrl
 * @return {boolean}
 * @private
 */
function isFresh(url, parsedUrl) {
  return (
    typeof parsedUrl === 'object' &&
    parsedUrl !== null &&
    parsedUrl._raw === url
  );
}

const req = {
  url: "localhost:8000/foo%2Fbar/baz/ąę"
}
console.log(parseurl(req).path)

Result
{
  href: 'localhost:8000/foo%2Fbar/baz/ąę',
  path: '8000/foo%2Fbar/baz/%C4%85%C4%99',
  pathname: '8000/foo%2Fbar/baz/%C4%85%C4%99',
  search: '',
  query: '',
  _raw: 'localhost:8000/foo%2Fbar/baz/ąę'
}

@fzn0x Thanks for the suggestion!
First of all, I don't think the cache will speed up the process, since we won't be calling getPath() multiple times for the same request object.
Also, I don't think the code you suggested would perform well if it were not cached.
If you still think it performs better, I would appreciate it if you could suggest it along with a benchmark.

FYI
The results of using symbols and multibyte characters in the routing definitions were investigated in the major frameworks. They vary considerably from framework to framework.
I don't think "we should match the major frameworks," but just that "if symbols and multibyte characters are not well supported in the major frameworks, it indicates that no one is having trouble in the production environment.
express
Symbols and multibytes do not appear to be available. paths are treated as "/"-separated segments.
const express = require('express')
const app = express()
const port = 3333

// "|" has a special meaning, so use "^" to validate
app.get('/^hello^', (req, res) => {
  res.send('^hello^')
})

app.get('/hello🔥', (req, res) => {
  res.send('hello🔥')
})

app.get('/users/:id/action', (req, res) => {
  res.send('users action')
})

app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})

$ curl 'http://localhost:3333/^hello^'
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /%5Ehello%5E</pre>
</body>
</html>
$ curl 'http://localhost:3333/%5Ehello%5E'
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /%5Ehello%5E</pre>
</body>
</html>
$ curl 'http://localhost:3333/hello🔥'
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /hello%f0%9f%94%a5</pre>
</body>
</html>
$ curl 'http://localhost:3333/hello%f0%9f%94%a5'
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error</title>
</head>
<body>
<pre>Cannot GET /hello%f0%9f%94%a5</pre>
</body>
</html>
$ curl 'http://localhost:3333/users/1%2F2/action' # 200 : id => 1/2
$ curl 'http://localhost:3333/users%2F1%2Faction' # 404

ruby on rails
Only percent-encoded URLs are supported; unencoded URLs will result in a 404. paths are treated as "/"-separated segments.
get '/^hello^'
get '/|hello|'
get '/hello🔥'
get '/users/:id/action'

$ curl 'http://localhost/^hello^'            # 404
$ curl 'http://localhost/%5Ehello%5E' # 200
$ curl 'http://localhost/|hello|'               # 404
$ curl 'http://localhost/%7Chello%7C' # 200
$ curl 'http://localhost/hello🔥'             # 404
$ curl 'http://localhost/hello%F0%9F%94%A5' # 200
$  curl 'http://localhost/users/1%2F2/action' # 200
1/2
$  curl 'http://localhost/users%2F1%2Faction' # 200

dgango
Before routing, all characters are decoded in a decodeURIComponent() equivalent process, showing similar results to https://github.com/honojs/hono/pull/2688.
path('^hello^', views.index, name='test'),
path('|hello|', views.index, name='test'),
path('hello🔥', views.index, name='test'),
path('a/<slug:slug>/c', views.path, name='test'),

$ curl 'http://localhost/^hello^'            # 200
$ curl 'http://localhost/%5Ehello%5E' # 200
$ curl 'http://localhost/|hello|'               # 200
$ curl 'http://localhost/%7Chello%7C' # 200
$ curl 'http://localhost/hello🔥'             # 200
$ curl 'http://localhost/hello%F0%9F%94%A5' # 200
$ curl 'http://localhost:8000/a%2Fb%2Fc' # 200
slug = b
$ curl 'http://localhost:8000/a/a%2Fb%2Fc/c' # 404

If what we need is a result similar to Ruby on Rails, then the following changes will accomplish this. (Actually, it's a bit more complicated, but it's "encode when adding a routing.")
diff --git a/src/hono-base.ts b/src/hono-base.ts
index 293d3874..d4cb8071 100644
--- a/src/hono-base.ts
+++ b/src/hono-base.ts
@@ -290,7 +290,7 @@ class Hono<

   private addRoute(method: string, path: string, handler: H) {
     method = method.toUpperCase()
-    path = mergePath(this._basePath, path)
+    path = encodeURI(mergePath(this._basePath, path))
     const r: RouterRoute = { path: path, method: method, handler: handler }
     this.router.add(method, path, [handler, r])
     this.routes.push(r)

If what we need is a result similar to Ruby on Rails, then the following changes will accomplish this. (Actually, it's a bit more complicated, but it's "encode when adding a routing.")
diff --git a/src/hono-base.ts b/src/hono-base.ts
index 293d3874..d4cb8071 100644
--- a/src/hono-base.ts
+++ b/src/hono-base.ts
@@ -290,7 +290,7 @@ class Hono<

   private addRoute(method: string, path: string, handler: H) {
     method = method.toUpperCase()
-    path = mergePath(this._basePath, path)
+    path = encodeURI(mergePath(this._basePath, path))
     const r: RouterRoute = { path: path, method: method, handler: handler }
     this.router.add(method, path, [handler, r])
     this.routes.push(r)

Yes, that's what I mean, sorry for the example, I think every framework does different things to deal with paths, but Django looks good!

If the expected behavior is equivalent to Ruby on Rails, I think #2711 can achieve this. I feel this is better than https://github.com/honojs/hono/pull/2688 because it is less burdensome at the request time.
However, I wonder if such routing is ever used in a production environment. If not, I think we can consider the option of "not supporting it".

I created #2714 with great support from @szmarczak!
Although #2711 is hard to discard for its performance advantage, at this point I consider this new #2714 to be the best.
The reasons are as follows

The path is divided into "/"-segmented segments, and the behavior is the same as the current version.
Prevent "intentionally slip through static paths using percent encoding" attacks.

https://github.com/usualoma/hono/blob/54cad32a63e5205b700e09f280866e97f8e1e2ed/src/hono.test.ts#L759-L766

Although I am concerned about the following points.

However, I wonder if such routing is ever used in a production environment. If not, I think we can consider the option of "not supporting it".

Also, there is still the issue of how to handle URIError: URI malformed. (I think it should return an error.)
@yusukebe Looking at the discussion so far, what are your thoughts?

@szmarczak @usualoma @fzn0x
Thank you for the discussion, and I'm sorry for the misunderstanding at first.
I like #2714! It achieves ideal handling paths, and the performance is good.

URIError: URI malformed

I also think it should return an error. When should it return the error? Can it return an error during the registration phase?

@usualoma If someone sends UTF-8 (non-ASCII) path over the wire, it's incorrect as the URI spec permits only ASCII. UTF-8 characters must be percent-encoded. So the Ruby on Rails behavior seems most correct.

Although https://github.com/honojs/hono/pull/2711 is hard to discard for its performance advantage, at this point I consider this new https://github.com/honojs/hono/pull/2714 to be the best.

The performance should be the same. The least performant way is when someone sends %25 which is 1 of dozen characters that have to be percent-encoded, so the probability of this path is very low.

Also, there is still the issue of how to handle URIError: URI malformed. (I think it should return an error.)

Well, (unfortunately!) the URI spec does not define the encoding percent-encoded characters have to use. It's just that the browser APIs like decodeURI and encodeURI force UTF-8 (using system encoding does not make sense as there are 8 billion people in the world with varying languages, so they had to arbitrarily define the encoding as UTF-8 (like MSL in TCP)).
Whether you return 400 Bad Request or use the URI without decoding in that case (it failed to decode to UTF-8) - any approach would be ok IMO.

However, I wonder if such routing is ever used in a production environment. If not, I think we can consider the option of "not supporting it".

Having developed a Node.js HTTP client (sorry for bragging!), I can definitely say that people widely send UTF-8 encoded characters (well, some even send non-UTF-8 but that's minority and we were enforcing UTF-8 for years (we had to disable UTF-8 enforcing in order to support legacy servers using other encodings).

Difference in performance between #2711 and #2714
I think the difference between the two at request time is indicated by the following part of the benchmark: for bun, #2714 is slightly faster; for deno and node, #2711 is faster. (This is a short URL benchmark, not including the %)
But in any case, it's a small difference, so I wouldn't worry too much about it.

Who is affected by this modification?

However, I wonder if such routing is ever used in a production environment. If not, I think we can consider the option of "not supporting it".

Even<EMAIL_ADDRESS>can process URLs containing percent encodings with the following routing
app.get('/users/:id', (c) => c.text(`id is ${c.req.param('id')}`))

$ curl 'http://localhost:3000/users/|ok|'
id is |ok|
$ curl 'http://localhost:3000/users/%7Cok%7C'
id is |ok|

What cannot be handled correctly is the case where "the routing definition contains symbols or multibyte characters.
app.get('/|', async (c) => c.text('ayy'));

I would like to merge #2714 because it is more correct for the framework to be able to handle these cases. However, I don't think there are any realistic users who would define such a routing for a web service endpoint that is widely accessed by various clients.
Handle "URIError: URI malformed"
@szmarczak
Sorry for not being clear on the subject. You were primarily targeting "errors that cannot be decoded as UTF-8", while I believe I was primarily targeting "incorrect percent encoding errors".

decodeURI("%A4%A2") : Japanese "あ" encoded in EUC-JP
decodeURI("%a") : Invalid encoding

You are right, regarding the former, I think it would be best if the framework does not make it an error, but in a way that the user can handle it himself.
As for the latter, I think it should return 400 as soon as possible.
Of course, getPath() could be modified to handle these properly, and hono-base could be rewritten, but I don't want to make getPath() too large.
I am considering this.

However, I don't think there are any realistic users who would define such a routing for a web service endpoint that is widely accessed by various clients.

It depends what you're developing. If you're developing an HTTP client, then you need to have a spec-compliant server to properly test the client. Or you just need static routes for simple mock server.
Also, being a Polish guy, I can say there are websites exposing static routes like /główna which means /home in English. The fix would allow these cases, but more importantly would be spec-compliant.

decodeURI("%a") : Invalid encoding

I wouldn't worry about this. First, it's against the spec. Second, it's impossible to get this via HTML forms (because forms are spec compliant). Proper fetch usage using WHATWG URLs never results in malformed URIs. The only possible situation for this is when someone intentionally types that.

@szmarczak Thank you!

more importantly would be spec-compliant.

I agree with you on that point.

decodeURI("%a") : Invalid encoding

I wouldn't worry about this.

Yes, of course, a user trying to send a correct request would not send the /%a/ path. However, there may be clients who send it for an attack.
Suppose we have a routing definition like this,
app.get('/content/:path', (c) => c.text('content'))
app.get('*', (c) => fallback()))

And suppose we have the following URL
http://localhost/users/%63ontent/path-to-%a

I think it would be best if a request to this URL would result in an error before entering the router, rather than proceeding to fallback.

Prevent "intentionally slip through static paths using percent encoding" attacks.

As I said in a previous comment, I think #2714 has the goodness to prevent this. If this feature is compromised, I think #2711 would be sufficient.
About #2711
As shown in the following test, I would expect #2711 to return the expected result for processing /| (or /główna), but would you prefer to choose #2714 over #2711?
https://github.com/honojs/hono/pull/2711/files#diff-8ac13809c9886e994d1db33943de82df4d6c5d88b73fd0270c0189804ff565c2R745-R765

Oh, by the way, there is another issue that is difficult to handle in #2714 in terms of spec-compliant.
Suppose we have a routing definition like this,
app.get('/główna/:path', (c) => c.text('główna'))

And suppose we have the following URL
http://localhost/g%C5%82%C3%B3wna/%A4%A2 # %A4%A2 is not a UTF-8 character

I think this is a spec-compliant URL, but it is difficult to get it routed in #2714 (while maintaining the spec of accepting non-UTF-8 encodings). #2711 would accomplish this.

I've come to the conclusion that with a few adjustments #2714 should be fine. Wait a minute.

Yes, of course, a user trying to send a correct request would not send the /%a/ path. However, there may be clients who send it for an attack.

That's a very good find! In this case it makes more sense to return 400 Bad Request.

would you prefer to choose https://github.com/honojs/hono/pull/2714 over https://github.com/honojs/hono/pull/2711?

I believe there's no difference in terms of the result. I'd choose what's more performant (or maintainable) and link to the other possible solution if you ever needed to change the algorithm.

I think this is a spec-compliant URL

It is!

it is difficult to get it routed in https://github.com/honojs/hono/pull/2714 (while maintaining the spec of accepting non-UTF-8 encodings)

The main branch is affected as well. The spec tells you that the percent encoded characters must be decoded, but it doesn't specify the encoding (making it a Uint8Array is unusable, if anybody wants to send binary data they're better using POST). Browsers send UTF-8, so I'd force it to be UTF-8 and 400 Bad Request if it isn't (I'd leave decodeURIComponent as is).

I'm not sure about not decoding non-UTF-8 characters in #2714, but I'm leaning towards being okay.
I'm satisfied with #2714 👍🏼

@szmarczak Thanks right away! I was going to comment on this, but I'm late.
Added process for invalid percent-encoding and invalid UTF-8 sequences.
Skip these sequences and apply decodeURI() to the rest as much as possible.
https://github.com/honojs/hono/pull/2714/commits/d252d13f2a64bcc90e0a72e508607a7d10feba51
This will prevent the following URLs from also slipping through the routing
http://localhost/users/%63ontent/path-to-%a
http://localhost/g%C5%82%C3%B3wna/%A4%A2

(As was the case before this PR.) If an encoding other than UTF-8 is used, a URIError is thrown when trying to retrieve it with c.req.param(). If you want to get it, you can get it by yourself from c.req.url, or write your own getPath().
https://github.com/usualoma/hono/blob/ea8ec95852aca34a1ed06177c8202e5b7e550404/src/hono.test.ts#L766-L809
Invalid percent encodings, such as %a, may be allowed to exit with a 400 error before invoking handlers, but that error checking is beyond the scope of this issue and is not addressed here.

@szmarczak Thanks again for all the thought-provoking comments so far, and for your thoughtful answers to all my questions.
#2714 or #2711
I think #2711 is often marginally better in terms of performance.
However, #2714 has more room for getPath() adjustments for users who want to do more complex things, and (not mentioned so far) regular expressions such as the following can be applied to decoded strings.
app.get('/:path{home|główna|home}')

I think #2714 is a good choice because of these various good points.

@yusukebe How about we make this issue resolved by #2714, including error handling?
@fzn0x @mgrithm
If you have any thoughts, feel free to comment.

@yusukebe
Added "Specification Changes" item to descriptioin. #2714

@usualoma
It looks good to me! The implementation of tryDecodeURI() and using app.onError for URIError is good.
Shall we go with it?

@yusukebe Thanks for the confirmation.
Let's go with it!

Let's go with it. I don't have any suggestions; I trust everyone here. :)


@@resource not scoped

Hello,
First, thanks for your project !
I would like to implement something similar in a complex redux app, but I'm stucked with your "Scoped types" example.
You write:
/*
scopedTypes == {
  "TOGGLE_ASIDE": "@@list/DEVICE/TOGGLE_ASIDE",
  "FETCH_USERS": "@@resource/USER/FETCH"
}
*/

Your USER/FETCH action is not scoped. I guess it's because it's a common action to fetch a resource, so you want to use the same action for the whole app.
Actually, I also would like to create a "not scoped" action (or at least a "special resource scope") for such actions.
Can you explain how you do that ? What's your solution to use a kind of global actions like @@resource in your project ?
Thanks for your help

The Readme is actually incorrect on this one, there should not be a FETCH_USERS key there. I'll update the docs.
Scoping is opt-in so once you have a working store with types & actions, you can if you want override them with an isolated scope.
If you want to use non-scoped actions along your newly scoped ones, you should use combineReducers from redux:
const rootReducer = combineReducers({
  generations: combineReducers({
    resource: generationsStoreModule.reducers
  }),
  list: scopedListStoreModule.reducers
});


bugfix: select from table external stream gets stuck if a topic contains invalid data

PR checklist:

Did you run ClangFormat ?
Did you separate headers to a different section in existing community code base ?
Did you surround proton: starts/ends for new code in existing community code base ?

Please write user-readable short description of the changes:
fixes #765

(Jove Github Bot) added it to the current sprint.

(Jove Github Bot) moved this ticket out of the GitHub project(up to 1200 tickets for one project).


[abc] Youtube embed results in "Unable to extract video urls" error

$ youtube-dl --verbose  http://www.abc.net.au/news/2015-08-17/warren-entsch-introduces-same-sex-marriage-bill/6702326
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['--verbose', 'http://www.abc.net.au/news/2015-08-17/warren-entsch-introduces-same-sex-marriage-bill/6702326']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2<IP_ADDRESS>
[debug] Python version 3.4.3 - Linux-4.0.7-2-ARCH-x86_64-with-arch
[debug] exe versions: ffmpeg 2.7.1, ffprobe 2.7.1, rtmpdump 2.4
[debug] Proxy map: {}
[abc.net.au] 6702326: Downloading webpage
ERROR: Unable to extract video urls; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 655, in extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 287, in extract
    return self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/abc.py", line 30, in _real_extract
    flags=re.DOTALL)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 569, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract video urls; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

Thanks for the report and sorry for the late response. YouTube videos on abc.net.au will be supported in the next version.

@stepshal Could you open a new issue?


add leaderboard support for misc_support.csv

@lehnberg Resolves #115.
Also, site will not break if this PR is merged before mimblewimble/grin-pm#89.
I.e. misc_support.csv will be ignored if it's unavailable.

Awesome work, thanks for the quick fix! Seems to be working just fine when I tested it now. ^^


Provide basic removal function

Currently the modul only allows to create new services. There is no easy solution to remove the services again.

Thanks for your contribution, I've merged it on PR-150


Alert on invalid configuration reload

Hi guys,
One of my pain point with Traefik is there is no alerting method for failed configuration reload.
I mean, my Traefik instance can watch consul updates, but, i have no way to detect that the new configuration is invalid.
The load balancer being the entrypoint of the whole stack, it is a huge SPOF if a consul service block the deployment of any other service because of an invalid config.
It’s probably not the best method, but what about making a Twilio or SMTP request ? Starting with a flag in Traefik metrics could also be great.

What do you mean by invalid config ?

For example, using the consul provider, if you accidentally set a tag to "traefik.backend.weight=foobar" in the consul catalog, traefik will throw an error "Failed to create config" [...] "Skipping empty configuration".

We could add this to traefik.toml:
onConfigError = "POST http://whatever.io/path"
Sending a post request with message in body, to whatever.io/path when configuration fail to update.
Then, it can be parsed and redirected by zapier, to Twilio, SendGrid...

@samber: I think this is a bit too specific for traefik.
This should rather be delegated to a monitoring app like prometheus.
For example, you could send an email at each ERROR log.
But the best solution would be to get this info in /health traefik API in the futur.
WDYT?

Yes, prometheus looks good, that's why we should set a counter in Traefik health api to allow Prometheus to detect errors.
Unfortunately, for people not using prometheus, there is still no solution.

Closing this one for now, as I think that monitoring tools should be used in that case.

How do you monitor that ?

You can monitor ERROR logs.

Hi,
@emilevauge your idea to get this information from /health sounds good as it would be easy to any monitoring software be aware of reload issues.
Do you guys plan to add this feature soon ?
Monitoring error log is not the right solution imho...
Thanks.


What normal losses value should be

Hi, Thanks for your work and sharing code. I tried to train the cdfse model with aishell3 dataset but got  abnormal synthesis result without playing sound and a very low classify accuracy value as follows finally.
Step 474000/500000, Total Loss: 11.3225, Mel Loss: 0.4685, Mel PostNet Loss: 0.4679, Pitch Loss: 0.0482, Energy Loss: 0.0543, Duration Loss: 0.0333, Cls Loss: 5.8576, Cls acc: 0.0400
Could you help me figured out which wrong step maybe leads in this result and what should the normal losses value looks like?

Much appreciated for your quick reply and help. The classify weight is 1.0 kept unchanged as default value. I used a different hifigan config model which may lead to the weird result. I will train a new hifigan and take a new experiment. Learns a lot from your repos, thank you again!

Much appreciated for your quick reply and help. The classify weight is 1.0 kept unchanged as default value. I used a different hifigan config model which may lead to the weird result. I will train a new hifigan and take a new experiment. Learns a lot from your repos, thank you again!

I found that this error might be due to the code in text/symbols.py (we neglected to update it in past commits).
If we run the code in AISHELL3, the symbol dict is best to exclude inrelevant phonemes (like English alphbets or others), otherwise it will greatly increase the difficulty of phoneme classification and cause this loss fail. Hope that solves your problem.

Thank you @Labmem-Zhouyx , I retrain the model and get the loss roughly the same as you showed above. I add some custom voice data for synthesis, and it seems that the speaker voice similarity drops down largely when the speaker is not in training set, while the content phoneme  features remains as good as those speaker in training data. Would you mind to leave a connection manner or add my QQ:359974136 for discussing?

It's a nice question. If the unseen speaker's voice is significantly different from the AISHELL3 dataset, the timbre similarity may drop largely because we have not made careful designs (like introducing GE2E loss, pre-trained Speaker Encoder on a very large speaker dataset, Contrastive Learning, etc.) in this regard. As shown in the paper, our method mainly aims to transfer personal pronunciation characteristics related to phoneme content to improve the speaker similarity from the sense of listening, and it seems to work in most cases.
The phenomenon you mentioned is the major problem faced by the current one-shot/zero-shot TTS system, that is, there is a domain difference between the training set and the unseen speaker, which also exists in GSE, CLS, or other methods.
To further improve the speaker similarity for unseen speakers, I think there are the following possible ways:

Incorporate some careful designs mentioned above to improve the generalization of speaker embedding.
Do model adaptation (fine-tuning the model with speech data of the target speaker).

By the way, your QQ is unable to be searched. If you are interested in further discussion, you can send emails to<EMAIL_ADDRESS>：)


Explicit permissions in operator ClusterRole

SEC-OPS-DEP-04
recommends explicitly enumerating permissions in an RBAC (Cluster)Role
rather than using wildcards ("*"). We believe wildcarding permissions
for groups we own is appropriate; but fix it elsewhere (which turns out
to be only one place).
HIVE-1919

/assing @akhil-rane

/lgtm


Not working with http server

I'm trying to use reflex to compile, stop and then restart a http server written in Go. But after http server listening in some port, suppose :8080 the reflex does nothing. In a simple hello-world function works fine, but after start a http server not working.
I have a Make file:
build:
	GOOS=linux go build -o ./build/api ./main.go

start:
	make build
	echo "Starting build/api"
	./build/api
	reflex -r \.go make restart

restart: stop build start

stop:
	echo "Stopping ./build/api if it's running"
	kill -9 `cat ./tmp/.pid`

So I just need to run make start and server will start fine, but never will execute the line reflex -r \.go make restart.
The .pid file on stop command is generated by my go program before start the server. And is working fine. Just the restart command is never executed by reflex.
Someone already had this behavior before?

This is an issue with your Makefile, not a reflex problem. The make process blocks on running ./build/api and never runs reflex. You might be able to fix it by starting up the server with ./build/api &.
But in general I'd advise against using PID files (for this or any other purpose).
Reflex has --start-service mode specifically for these server uses cases.

@cespare Thanks for your reply and advice about PID files.
I changed my make start to:
start:
	make build
	echo "Starting build/api"
	./build/api&
	reflex --start-service true -r \.go make start

And change a log line in my code just to test and then the output on the cli was something like that:
[00] make[1]: Nothing to be done for 'restart', the server still running but not with the new changes.

This is still makefile problems, not anything to do with reflex.
Google "makefile phony".

I understand now. Thanks.

I rewrite my make start to:
start:
	reflex --start-service true -r \.go
	make build
	./build/api

build:
	GOOS=linux go build -o ./build/api ./main.go

.PHONY: start build

But not works. My program start with the http server as well, but when I change from fmt.Println("test1") to fmt.Println("test2") the program shows "test1" yet on cli.
May could you provide a better example how to use the --start-service correctly to stop a http server, recompile the code and start the http server again?
Thanks in advance!

There are a few different issues here. One problem is that you need to write --start-service=true, or just --start-service. Another is that your start task blocks on reflex and never runs the other commands.
This should give you some ideas:
.PHONY: build run reflex

build:
	go build -o ./build/api ./main.go

run:
	./build/api

reflex:
	reflex --start-service -r '\.go$$' make build run


Flux.jl

Hi @ChrisRackauckas, I recently learned about the existence of the Flux.jl package.
I'm now looking at this package and have a question related to AD: It seems Flux.jl has its own AD; is this going to be supported or will it be replaced when a new one based on Cassette.jl is out? Thanks!

Yes, Flux.Tracker will continue to be supported. The more advanced ADs we are building will take some time to mature, so we won't be changing the default for a while yet.
The plan is that the current and future ADs will be API compatible, with the only real difference being that the current data and param functions become no-ops. Code using the current AD should be able to update or try out the new one with a one-line change, so there's pretty low risk on depending on it.
Happy to answer any other questions that would help you get going with it.

@MikeInnes Many thanks.
At the moment, Turing makes use of ForwardDiff.jl and ReverseDIff for AD.  We are considering to switch to Capstan when it's more mature.
It would be nice if Flux's AD can stay API compatiable with Capstan. We might consider to switch earlier if it represents Julia's future AD. What are the advantages of Flux.jl's AD engine over these existing ones?

The main reason Flux has its own AD is in order to support modularity in parameters (discussed at length in https://github.com/denizyuret/Knet.jl/issues/144). Other than that, things like performance characteristics are broadly similar.

The main reason Flux has its own AD is in order to support modularity in parameters (discussed at length in denizyuret/Knet.jl#144). Other than that, things like performance characteristics are broadly similar.

We struggled with the same issue. Basically, Turing models can introduce new variables/parameters anywhere inside the @model block. This means before running the model function, we know nothing about

the total number of parameters
the dimension of each parameter
the data type of each parameter

@xukai92 managed to hack the ForwardDiff.jl and ReverseDiff.jl in order to the support the Turing modelling syntax. The implementation involves several runs of the model function to prepare the AD. Therefore it would be great if the modularity support from Flux can be incorporated into Turing.


troubleshooting documentation changes

@kyrylo only digma.environment.id should be documented !!
no need to mention digma.environment or digma.user.id settings
digma.environment.id=[ENVIRONMENT_ID]
@ionuiux @kshmidt
we need to point the user from where to take the environment id (clicking on the environment hamburger menu ...) , also screenshot will help here

close this one when done - https://github.com/digma-ai/digma-intellij-plugin/issues/2081

https://www.figma.com/file/MxA95ZHK5NqGX4ptFQljRJ/⭐-Design-System-3.0?type=design&node-id=6132-115942&mode=design&t=n7ONDTFaAq2Bxfs1-11 - Design Link


Added the removal of a columns default constraint, before dropping the column

PR targeting issue #2492, which is specific to MSSQL.

@elhigu Unit tests updated; ready for review again; let us know if anything else needs to be changed/you have suggestions; thank you!

@tetsupanda Would you consider finishing this?

Any progress?

I copied the changes into my local node_modules/knex and it works fine for:
"mysql": "2.15.0"
"knex": "0.19.5",

Superseded by #3855


"The system is busy"

When trying to place an order, I get this:
{'success': False, 'code': 'trade.system.exception', 'msg': 'The system is busy', 'data': {'lastSerialId': '5f0c8537ca18e90001147115'}}
Is this just due to webull servers being overloaded or am I facing locked account issues?

Sorry, I messed up and forgot to input a quantity. I'm guessing this shows up if you place an order with a quantity of 0.


ee stack restart doesn't consider remote mysql

> ee stack restart
Restart : nginx     [OK]
Restart : php5-fpm  [OK]
Restart : mysql     [Failed]
Restart : postfix   [OK]

@gau1991 As we discuss earlier we have to check whether mysql-server or percona-server-server package is installed or not.
Also you can check /var/lib/mysql directory to cross check mysql is local or remote.

I think /var/lib/mysql is not foolproof. MySQL config must have way to override storage location.

Fixed with EasyEngine 3.0.3


Update datasets to 2.13.0

https://github.com/huggingface/datasets/releases/tag/2.13.0
Related to the datasets server:

Better row group size in push_to_hub by @lhoestq in https://github.com/huggingface/datasets/pull/5935
Make get_from_cache use custom temp filename that is locked by @albertvillanova in https://github.com/huggingface/datasets/pull/5938

Did I miss something @huggingface/datasets-server ?

it should also fix issues where images were shown as {"bytes": ...} thanks to https://github.com/huggingface/datasets/pull/5921


WIP: Add AppImage workflow

Summary
As discussed in #916, I see value in a release of ArchiveBox as an AppImage along with the other releases.
Therefore, this commit adds a github workflow for automatically building an AppImage on pushes.
Please consider this PR as work in progress. The AppImage building works, but I'm not 100% sure as to how you wish to exactly integrate this workflow into the repository and how uploading to the releases page works. Currently, the AppImage is uploaded as an artifact of the CI-run to show that working artifacts are produced. Currently, you seem to have factored out all distribution-methods into their own submodules. Should I do the same for the appimage_dist?
Furthermore, I plan to write a short bit of documentation on the specifics of the AppImage (i.e.: what is packaged, what is not, that there is no sandboxing and that other distribution methods are preferred and potentially better supported). I'd probably put that into docs/Install.md. Does that sound like the correct location?
Any other comments/requests/*?
Thanks for your efforts!
Related issues
#916

Changes these areas

[ ] Bugfixes
[ ] Feature behavior
[ ] Command line interface
[ ] Configuration options
[ ] Internal architecture
[ ] Snapshot data layout on disk

Unfortunately I don't have the bandwidth to support another release channel myself, but if you want to create a separate repo that releases ArchiveBox as an appimage, I can link to it from our README.


Emptry thread startpoint and thread endpoint in Ladybug report for XsltPipe

Describe the issue
When running an XsltPipe an emptry thread startpoint and and empty thread endpoint appears in the Ladybug report.
Reporter
Jaco de Groot

To Reproduce
Steps to reproduce the behavior:

Start iaf-webapp
Goto test a pipeline and run ManageDatabase with the input mentioned below
Open the report in Ladybug and see the following:

Screenshots
See above
Configuration
-
Input
<manageDatabaseREQ>
	<select>
		<tableName>IBISSTORE</tableName>
	</select>
</manageDatabaseREQ>

Environment
FF! 7.7-SNAPSHOT: webapp
Running on LPZWNL000040359 using Apache Tomcat/9.0.50
Java Version: OpenJDK Runtime Environment (1.8.0_292-b10)
Heap size: 167M, total JVM memory: 723M
Free disk space: 189GB, total disk space: 475GB
Up since: 2021-08-06 18:45:06 (7m)

Additional Environment
-
Additional Context
-

The 'empty' thread is the thread in which the Xslt transformation runs. At the time it is created, it is unknown that no further events will be displayed in it.


refactor: remove useless CachedProperty

This is a breaking change: this PR removes the @CachedProperty decorator because it was mainly used to merge directives in a widget before using them with auUse (cf this commit), which only supports a single directive. Now that auUseMulti is available to apply multiple directives without requiring to merge them before, CachedProperty is no longer needed most of the time.

@quentinderoubaix Thank you for your review!


luci-proto-bonding: ethernet bonding interface (device) dhcp client functionality hindered (re-opened)

Per- https://github.com/openwrt/packages/issues/23677#issuecomment-2002278900 - re-opening this to track the issue/fix on the openwrt/luci end of things.
Steps to reproduce:

go to: Network → Interfaces → Add new interface...  or bond0 (a Link Aggregation (Channel Bonding) interface) → Advanced Settings
add one or more Slave Interfaces
Set Bonding Policy to IEEE 802.3ad Dynamic link aggregation (802.3ad, 4)
go to: General Settings tab
clear IPv4 address and IPv4 netmask
Save

Actual behaviour:

Expecting: non-empty value
Unable to create a Layer 2 bonding interface which can directly requisition an IP via DHCP
The link aggregation device must be configured with a dummy non-colliding address (such as a loopback address) which serves no purpose/has potential to cause tangential issues which would otherwise be unnecessary (with regards to the Linux bonding driver itself)
A parent bridge device containing the subsequently generated bond-bond0 device, as expected, can be configured as part of a DHCP client interface, rendering the mandated dummy static IP assigned to the bond interface entirely redundant

Expected behavior:
A bonding Device (not an Interface, per the LuCI abstraction) can be created directly, without the requirement of a dummy static IP, and have its address managed with any combination of:

as part of a Static address interface
as part of a DHCP client interface
as part of a parent bridge device, which itself is managed with a DHCP client interface

Additional Information:

using DHCP for a bonded interface (per the Linux kernel terminology) is supported (but requires active slaves, of course; something has to be able to broadcast the DHCP request)
most tools in the Linux ecosystem have supported DHCP for bonded devices since at least 2004
for example, netplan does not require either static or DHCP addresses to be configured, but also accepts and works perfectly fine with either approaches

NAME="OpenWrt"
VERSION="SNAPSHOT"
ID="openwrt"
ID_LIKE="lede openwrt"
PRETTY_NAME="OpenWrt SNAPSHOT"
VERSION_ID="snapshot"
HOME_URL="https://openwrt.org/"
BUG_URL="https://bugs.openwrt.org/"
SUPPORT_URL="https://forum.openwrt.org/"
BUILD_ID="r25529-1d3d6ef826"
OPENWRT_BOARD="mediatek/filogic"
OPENWRT_ARCH="aarch64_cortex-a53"
OPENWRT_TAINTS=""
OPENWRT_DEVICE_MANUFACTURER="OpenWrt"
OPENWRT_DEVICE_MANUFACTURER_URL="https://openwrt.org/"
OPENWRT_DEVICE_PRODUCT="Generic"
OPENWRT_DEVICE_REVISION="v0"
OPENWRT_RELEASE="OpenWrt SNAPSHOT r25529-1d3d6ef826"

On the openwrt/luci / luci-proto-bonding end of things, this limitation is enforced here
On the openwrt/packages / proto-bonding end of things, this limitation is enforced here

Issue to track openwrt/luci is here
Issue to track openwrt/packages is here

I don’t think the luci-proto-bonding package will receive fixes to support non-static config, it should rather become deprecated in favor to bonding device config support which is present in netifd since https://github.com/openwrt/netifd/commit/5ba9744aac6d42da1e56357aca951b52f86cfacb
That also conceptually works better since bonding devices are a layer 2 config, not a layer 3 (proto) one.


Changes to support withCredentials attribute of XmlHttpRequest object

This pull request is to add support for allowing a new withCredentials attribute to the ember plupload component.  It requires changes requested in pull request to plupload https://github.com/moxiecode/plupload/pull/1356

Looks like Plupload rejected the other side of this. Sorry!


Add ability to create a Cassandra cluster extended on multiple k8s clusters

Another ticket to be CassKop compatible. This ticket is to have something similar to multiCassKop. So it's different from #318 as it is about extending a cluster on multiple kubernetes clusters. Of course it also needs to work with multiple datacenters, so it would probably make sense to implement it after #318.

Migrated to https://github.com/k8ssandra/k8ssandra/issues/943


Qualified table name support

I was wondering would it make sense to support specifying schema (qualified table name) while querying?
For example I have a PostgresDb with schema created for every user (multi tenancy), and I don't see any simple way for querying using qualified table name or querying across schemas. The same goes for MySql and querying across databases.

I solved this problem this way:
type Person struct {
  ...
}

func (this *Person) TableName() string {
    return "dw.Person"
}

Although I'd prefer to have something like that:
func (this *Person) SchemaName() string {
    return "dw"
}

TableName method might work fine to dynamicaly create schema suffix. I will try that

Yes, you could use TableName, or like db.Table("tx.users").Find(&user)


Can't unlock if fingerprint fails

Describe the bug
There is no fallback to PIN when the Unlock with the fingerprint reader fails.
To Reproduce

Open BBW
Attempt to unlock with and unregistered fingerprint a a fee times
Clicking unlock does not open anything afterwards

Expected behavior
Fall back to PIN if the fingerprint is not working.
Smartphone

Device: Google Pixel 3
OS: GrapheneOS
Version: Android 12

Additional context
App version: 2.2.13 (internal alpha)
Just pinging @UncleSamtoshi who I believe was last working on this.

Found the solution so closing.
Was able to reenable the fingerprint sensor after locking and unlocking the phone home screen with PIN -> the fingerprint unlock works again.
Found that the PIN Code was not active in the Security menu which was the reason it couldn't be used as a fallback.
Tested to disable the fingerprint sensor with wrong ties again and can confirm that if the PIN is set beforehand the app can be unlocked with PIN regardless of the sensor being disabled.

On a second thought in case the fingerprint sensor is permanently damaged would be good to have a PIN available.
Should we force the PIN to be created when enabling the Biometric login ?

On a second thought in case the fingerprint sensor (or finger) is permanently damaged would be good to have a PIN available otherwise the user can be locked out of the app.

feels an edge case that if we cover would 1/ increase complexity and 2/ add burden on the user? ie: like if the phone is broken, it's expected someone will have to log back in eventually?

Should the PIN creation be forced when enabling the Biometric login ?

On a second thought in case the fingerprint sensor (or finger) is permanently damaged would be good to have a PIN available otherwise the user can be locked out of the app.
Should the PIN creation be forced when enabling the Biometric login ?

I am in favour of enforcing PIN creation when enabling Biometrics. Most fintech apps I've used have the PIN number pad overlaid in the background incase biometric fails. This would also be a good use case for people who have damaged biometric sensors (either Face ID or fingerprint).

yes, it is not just about the phone being damaged but the usecase eg. using glasses, hat, mask or gloves which would alter the biometrics even temporarily and cause inconvenience. Having a fallback is the usual practice.

#2960 Partially addresses this; at least makes the process smoother. If a user's biometric isn't working for whatever reason, there is the pin option (if enabled) or they can log out and log back in like they normally would. They can receive a code to their device or email


Remove requirement for totaltime metric

Looks like tests broke after the fix-daily-tests got merged, since there was a new metric 'totaltime' added, and we need to get rid of require for metrics now.

Thanks, sorry about that!


Sonarr Sensor - division by zero error

Make sure you are running the latest version of Home Assistant before reporting an issue.
You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:
Home Assistant release (hass --version): 0.53.0
Python release (python3 --version): 3.5.3
Component/platform: Sensor/Sonarr
I'm getting the following error when I start Home Assistant. Was happening on 0.52.1 also (hadn't used Sonarr before then).
2017-09-09 23:00:12 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/lib/python3.5/asyncio/tasks.py", line 239, in _step
    result = coro.send(None)
  File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/entity_component.py", line 381, in async_process_entity
    new_entity, self, update_before_add=update_before_add
  File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/entity_component.py", line 238, in async_add_entity
    yield from entity.async_update_ha_state()
  File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/helpers/entity.py", line 247, in async_update_ha_state
    device_attr = self.device_state_attributes
  File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/components/sensor/sonarr.py", line 138, in device_state_attributes
    )] = '{:.2f}%'.format(100*(1-(show['sizeleft']/show['size'])))
ZeroDivisionError: float division by zero

My configuration.yaml has the following for sonarr/radarr:
sensor:
  - platform: sonarr
    api_key: d48d00bd86b4429b9ce27e3b7cdc9a07
    host: server.home
    monitored_conditions:
      - series
      - upcoming
      - queue
      - diskspace
      - status
    days: 3
    urlbase: /sonarr
    include_paths:
      - M:\

  - platform: radarr
    api_key: 7227ae724fe24abd97e91d63af2cfe41
    host: server.home
    monitored_conditions:
      - movies
      - upcoming
      - diskspace
      - status
    days: 3
    urlbase: /radarr
    include_paths:
      - C:\

I suspect the problem is the the queue monitored condition as that's the only one not working - but is in the documentation.
I also noticed the queue condition isn't available in the Radarr component (according to the docs). And the status condition isn't in the Sonarr sensor docs (but works fine in 0.53.0).

I am having the same issue as of version 0.54.0. Would be nice to not have this always spamming my error logs.


Uncaught TypeError: Super expression must either be null or a function, not object

Duplicate of a closed issue. Though that post mentions updating a library to 2.0.0-rc2, it did not specify which library was to be updated.
I got everything updated to their latest version. Any help would be appreciated.

I actually managed to unknowingly replace 2.0.0-rc3 with 1.1.8 myself. Caused this exact issue


Start adding Schema descriptions to all Pydantic models in Fractal

Description
As it says in the title. Will be helpful for auto-doc generation
and future developers.
Still a work in progress. There is alot here though so I wanted to post
what I have as I keep editing. There are a few TODOs I have added
that I think need extra attention or that I have questions on.
Due to the large changes and the need for accuracy, I could use
some eyes on each of these to make sure I have described
each option correctly.
Provide a brief description of the PR's purpose here.
Todos
Notable points that this PR has either accomplished or will accomplish.

[x] Several outstanding TODOs I added need special attetion.

Status

[ ] Changelog updated
[x] Ready to go

I think this is ready. I noticed I left the TODO's, but that will be fix on next PR

We have decided to break this PR up into multiple PR's and handle entries in chunks.


BUG: Workaround for VariableLengthVector addition issue

Resolves PythonSpectra1DAveragingImageFilterTest failures where
adding itk::VariableLengthVectors with the addition operator +
would sometimes result in erroneous vector math values. The +=
operator is implemented independent of + for
itk::VariableLengthVectors and is observed to correctly execute
element-wise vector addition.

@dzenanz Also note that vector image baseline comparison does not seem to be working correctly as I was able to get passing tests for an all-zero vector image output that should have exceeded comparison tolerance. I have left baseline comparison active for now as it was useful for identifying wildly incorrect results from vector addition errors, but will better investigate the TestDriver logic in ITK proper to see if we need to make changes for vector image comparison.
I will also log an ITK issue for itk::VariableLengthVector::operator+ failures. I am fairly confident that this was the issue as I was able to observe bad vector math outputs from + and then successful vector math outputs from += via inline function printouts . The + operator attempts to make use of metaprogramming techniques for execution speedup, which theoretically offers advantages but seems to have introduced a bug here.

It looks like Windows Python CI failed with a bad dependency curl request:

curl: (92) HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)

May need to re-run CI.

Issue is here https://github.com/InsightSoftwareConsortium/ITK/issues/3218

MacOS errors seem to be in itkSpectra1DImageFilter and unrelated to this patch
https://open.cdash.org/viewBuildError.php?buildid=7754551

--compareIntensityTolerance is 2.0 by default. We should set it lower. After (or is it before?) --compare X Y add --compareIntensityTolerance 0.01.

@dzenanz I observed passing tests where the output was a 31-channel single-pixel image of all zeros while our test baseline is a 31-channel single-pixel image with channel values from 0.0 to 3.1. If the comparison should be elementwise by channel, then the comparison failed to catch values from 2.1 to 3.1 that should already have been out of tolerance. If the comparison is not intended to compare channels directly but instead some accumulated measure of pixel intensity, then we should investigate and determine what is the intended mechanism.

@dzenanz I do not have permission to re-run CI or merge the PR, would you prefer that we track down the comparison tolerance issue in this PR or is that okay to leave to a subsequent change?


Daniil: Watch on88play.net

Daniil requests the watch of the watch_keyword on88play\.net. See the MS search here and the Stack Exchange search here.

on88play\.net has been seen in 1 true positive, 0 false positives, and 0 NAAs.

Approved by tripleee in Charcoal HQ


Out of bounds checking for struct/union members

Currently, only constant indices are checked.

@jix This makes the tests you suggested in #3573 work.
I'm not sure how out of bounds checking should/could be done for dynamic indices - can a wire chunk representing a struct member somehow be broken out before the "shifter cell" in genrtlil.c is created?
I noticed that there is also a "replace dynamic ranges" in simplify.cc - this seems similar, but seems to be restricted to assignments.

@jix Struct/union members are now represented with wire chunks, and indexing is performed within these chunks.

@jix I just realized that read_verilog -dump_ast2 and -dump_vlog2 obviously don't take the wire chunk offset into consideration. Since this is a regression, I converted the PR to a draft for now.

@jix I think I'm happy with this now. No really intrusive changes, the chunks are visible in -dump_ast2 and -dump_vlog, and designs with dynamic indexing members which are part of bigger structs/unions can use less resources.


[qpid-proton] Fix missing runtime dependencies

Describe the pull request

What does your PR fix?
Fixes #24449

Which triplets are supported/not supported? Have you updated the CI baseline?

All, no.

Does your PR follow the maintainer guide?

Yes

Thanks!


Migrate primary branch from master to main

See https://github.com/Senzing/stream-loader/issues/283

Pull requests:

https://github.com/Senzing/evaluate/pull/38


One-to-many relationships can be captured using foreign keys?

How are foreign keys defined in IDL? What does it do?

Hi ZTY-star,
You can not define the foreign keys in IDL files. But you can use the foreign key concepts in your applications.

Hi ZTY-star,
You can not define the foreign keys in IDL files. But you can use the foreign key concepts in your applications.

Sorry, I do n’t understand. Can you give an example?

To understand relationship in DDS you need to understand following concepts:
Distributed Relational Information Modeling
– Topic Keys can be used to identify instances as well as relationships
– Relationships can be navigated by relying on a subset of SQL
– One-to-many relationships can be captured using foreign keys
– Many-to-many relationships need to be modeled using a topics
– Keys can be represented by an arbitrary number of Topic fields
Suppose you have 3 topics named Floor,Room and TempSensor. Topics types are given below:
Floor has a primary key fID. Room has a primary key rID and attribute fID is a foreign key. same in TempSensor, tID is a primary key and attribute rID acts as a foreign key.
struct Floor
{
short fID;
short level;
short open;
};
#pragma keylist Floor fID
struct Room
{
int rID;
float width;
float height;
float length;
short fId;  // Foreign Key
};
#pragma keylist Room rID
struct TempSensor
{
short tID;
float temp;
float huminity;
int rID;     // Foreign Key
};
#pragma keylist TempSensor tID
Now you need to maintain these relationship in your applications means you need to validate the received data as per the dependency and process accordingly.


Add crawler support for pagure.io

I am still pretty unfamiliar with ClearlyDefined so I hope this issue makes sense. Pagure.io is a hosted instance of pagure, an open source git-based source code hosting system associated with the Fedora Project, currently hosting over a thousand projects, some of which (admittedly not a huge number) are ultimately packaged commercially in Red Hat products.

Given that this issue has been transferred, I'm closing it.


How to add XMP metadata

Hello, for my project I need to add xmp metadata to my pdf.
In the doc I can only provide title, author ect to the pdf but not an xmp file.
How can I achieve that ? Thank you

pdfkit version: 0.15.1
Node version: 20.18.0
Operating System: Linux

I'm also trying to set up pdfkit properly. In my case to create a PDF/A-3b.
I found  https://github.com/PadraigGalvin/pdfkit-pdfa-sample/blob/master/generate.js
and in it there is a lot done going in the right direction. Maybe It helps you.
Can you tell us what exactly you need to do?

See documentation how to set PDF/A document (xmp metadata).


[Typography] Remove paragraph prop (use system instead)

A small documentation issue at page https://material-ui.com/api/typography/
At props section (https://material-ui.com/api/typography/#props): inside the table paragraph and gutterBottom both reports If true, the text will have a bottom margin. which is confusing.
I guess is the paragraph description that is wrong.

Edit by @oliviertassinari.
A proposal: we remove the paragraph prop as they are likely already covered by the system props, e.g. mb.

https://github.com/mui-org/material-ui/blob/cf75ed59666cb852f53f024ecaf57eb12fa64465/packages/material-ui/src/Typography/Typography.js#L72-L74
It also applies margin bottom, just a bit difference from what gutterBottom does.

This part of the Typography component is not pretty. I want to improve it since the release of v1. The documentation is not incorrect without being helpful. I would blame the implementation with the problem. I have personally almost never used the paragraph prop and use gutterBottom quite often.
What API would you like to see? As we are working toward moving the system into the core component, we should be able to leverage the m… props in the future.

To be honest I'm not using paragraph or gutterBottom neither. I just found this repetition by reading the whole docs looking for other.
I'm not sure if any API of this kind is required here.

@keul How are you handling the spacing between two different Typography elements?

I don't have much on them in my application right now, but I commonly use some className for this purpose

Ok, thanks for sharing.

I tend to use paragraph when I want to add spacing :/

Is there any reason for this component not to inherit the Box-style margin interface? I rarely use Typography because of this reason.

Is there any reason for this component not to inherit the Box-style margin interface? I rarely use Typography because of this reason.

@buchanaf What do you mean margin interface? Like <Typography m={2} />

@buchanaf What do you mean margin interface? Like <Typography m={2} />

@oliviertassinari, yes, exactly, including mx, my, etc.. Honestly, it'd be convenient for a lot of the more "primitive" components like Button.

@oliviertassinari, yes, exactly, including mx, my, etc.. Honestly, it'd be convenient for a lot of the more "primitive" components like Button.

@buchanaf <Button> already supports in v5 under the sx prop. For the Typography, Box, Grid, Stack, I have been advocating for having the subset of the system available as regular props.

@buchanaf <Button> already supports in v5 under the sx prop. For the Typography, Box, Grid, Stack, I have been advocating for having the subset of the system available as regular props.

Ah, good to know. I look forward to it! Good luck with the rest and thanks for your work!

Ah, good to know. I look forward to it! Good luck with the rest and thanks for your work!

We are deprecating the paragraph in Typography for v6. We'll iterate on https://github.com/mui/material-ui/pull/42383


UNIX socket path in node.js

Hi, is there a way for superagent to make a HTTP request using UNIX sockets instead of host:port?
Thanks!

I believe that may be technically possible but reviewing the current code I'm pretty sure it does not currently work. This would create another feature disparity between the browser and node, so there may be some objection to adding this based on that. My thought would be getting the basic mechanics working over a unix socket would be pretty straightforward but then it would get complex dealing with redirects and facets of HTTP coupled to the host:port origin like cookies etc.

Personally, I wouldn't be opposed to supporting that if it's just a few lines of code.

@enricostano
Hello! I've implemented it on https://github.com/kanreisa/superagent/commit/9e57664dc130d46fcc17a47301cf8756da339a55 .
If you sure, try it and feedback please.
Usage:
http://unix:/absolute/path/to/unix.socket:/request/path

hi @kanreisa thanks! But I finally decided to create 2 different and separate API clients, one for server side (using request) and another for browser side (using XHR, until fetch doesn't get stable enough).

@enricostano np ;)


Matter accessories appear unresponsive in HomeKit (CON-490)

Log output
E (2561939) chip[EM]: OnMessageReceived failed, err = 70
E (2574119) chip[EM]: OnMessageReceived failed, err = 70
E (2574179) chip[EM]: OnMessageReceived failed, err = 70
E (2599129) chip[EM]: Failed to Send CHIP MessageCounter:15364235 on exchange 53745i sendCount: 4 max retries: 4
E (2601539) chip[EM]: OnMessageReceived failed, err = 70
E (2601549) chip[EM]: OnMessageReceived failed, err = 70
E (2601819) chip[EM]: OnMessageReceived failed, err = 70

How do I locate and fix this bug

@shun-dgut This can be due to a variety of reasons. So, it is difficult to fix it for all the cases. The thing is, the device should still recover after this issue. Please let us know if that is not happening.
To give an example for this issue, in case of noisy networks where there are lot of retranmissions, it could happen that there are errors at the lwIP layer (like no memory) for sending the packets as the buffers get full. In such cases, if the upper layers handle retransmissions well then it would get resolved in any of the subsequent attempts

@shun-dgut If the device goes unresponsive, then please upload (and not paste) the device side log. In that case, I think this should be the issue and its fix: https://github.com/project-chip/connectedhomeip/pull/26215 .
We will soon update the connectedhomeip submodule to the next tag (when created) from the upstream with this fix.

@shun-dgut If the device goes unresponsive, then please upload (and not paste) the device side log. In that case, I think this should be the issue and its fix: project-chip/connectedhomeip#26215 .
We will soon update the connectedhomeip submodule to the next tag (when created) from the upstream with this fix.

Thanks，I am merging this for the test, and although the device will recover, it comes back after a while after recovery, and it will recover again.
My total stack is still with the remaining ones. I would like to open more logs to see.

@shun-dgut Please share your observations. We've now moved to connectedhomeip v1.1-branch which has this fix. Please close the issue if it works fine.


test_server_read_leader_with_unapplied_log failed

hread 'raftstore::test_multi::test_server_read_leader_with_unapplied_log' panicked at 'called Result::unwrap() on an Err value: Timeout("request timeout for 5s")', src/libcore/result.rs:1165:5
[2019-10-09T13:23:13.861Z] stack backtrace:
[2019-10-09T13:23:13.861Z]    0: backtrace::backtrace::libunwind::trace
[2019-10-09T13:23:13.861Z]              at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.35/src/backtrace/libunwind.rs:88
[2019-10-09T13:23:13.861Z]    1: backtrace::backtrace::trace_unsynchronized
[2019-10-09T13:23:13.861Z]              at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.35/src/backtrace/mod.rs:66
[2019-10-09T13:23:13.861Z]    2: std::sys_common::backtrace::_print
[2019-10-09T13:23:13.861Z]              at src/libstd/sys_common/backtrace.rs:47
[2019-10-09T13:23:13.861Z]    3: std::sys_common::backtrace::print
[2019-10-09T13:23:13.861Z]              at src/libstd/sys_common/backtrace.rs:36
[2019-10-09T13:23:13.861Z]    4: std::panicking::default_hook::{{closure}}
[2019-10-09T13:23:13.861Z]              at src/libstd/panicking.rs:200
[2019-10-09T13:23:13.861Z]    5: std::panicking::default_hook
[2019-10-09T13:23:13.861Z]              at src/libstd/panicking.rs:214
[2019-10-09T13:23:13.861Z]    6: std::panicking::rust_panic_with_hook
[2019-10-09T13:23:13.861Z]              at src/libstd/panicking.rs:477
[2019-10-09T13:23:13.861Z]    7: std::panicking::continue_panic_fmt
[2019-10-09T13:23:13.861Z]              at src/libstd/panicking.rs:384
[2019-10-09T13:23:13.861Z]    8: rust_begin_unwind
[2019-10-09T13:23:13.861Z]              at src/libstd/panicking.rs:311
[2019-10-09T13:23:13.861Z]    9: core::panicking::panic_fmt
[2019-10-09T13:23:13.861Z]              at src/libcore/panicking.rs:85
[2019-10-09T13:23:13.861Z]   10: core::result::unwrap_failed
[2019-10-09T13:23:13.861Z]              at src/libcore/result.rs:1165
[2019-10-09T13:23:13.861Z]   11: core::result::Result<T,E>::unwrap
[2019-10-09T13:23:13.861Z]              at /rustc/c6e9c76c59e3c10acd63ca9ec157a8894ea1a068/src/libcore/result.rs:933
[2019-10-09T13:23:13.861Z]   12: test_raftstore::cluster::Cluster::transfer_leader
[2019-10-09T13:23:13.861Z]              at ./components/test_raftstore/src/cluster.rs:796
[2019-10-09T13:23:13.861Z]   13: test_raftstore::cluster::Cluster::must_transfer_leader
[2019-10-09T13:23:13.861Z]              at ./components/test_raftstore/src/cluster.rs:817
[2019-10-09T13:23:13.861Z]   14: integrations::raftstore::test_multi::test_read_leader_with_unapplied_log
[2019-10-09T13:23:13.861Z]              at tests/integrations/raftstore/test_multi.rs:543
[2019-10-09T13:23:13.861Z]   15: integrations::raftstore::test_multi::test_server_read_leader_with_unapplied_log
[2019-10-09T13:23:13.861Z]              at tests/integrations/raftstore/test_multi.rs:573
[2019-10-09T13:23:13.861Z]   16: integrations::raftstore::test_multi::test_server_read_leader_with_unapplied_log::{{closure}}
[2019-10-09T13:23:13.861Z]              at tests/integrations/raftstore/test_multi.rs:571
[2019-10-09T13:23:13.861Z]   17: core::ops::function::FnOnce::call_once
[2019-10-09T13:23:13.861Z]              at /rustc/c6e9c76c59e3c10acd63ca9ec157a8894ea1a068/src/libcore/ops/function.rs:227
[2019-10-09T13:23:13.861Z]   18: <alloc::boxed::Box as core::ops::function::FnOnce>::call_once
[2019-10-09T13:23:13.861Z]              at /rustc/c6e9c76c59e3c10acd63ca9ec157a8894ea1a068/src/liballoc/boxed.rs:922
[2019-10-09T13:23:13.861Z]   19: __rust_maybe_catch_panic
[2019-10-09T13:23:13.861Z]              at src/libpanic_unwind/lib.rs:80
[2019-10-09T13:23:13.861Z]   20: std::panicking::try
[2019-10-09T13:23:13.861Z]              at /rustc/c6e9c76c59e3c10acd63ca9ec157a8894ea1a068/src/libstd/panicking.rs:275
[2019-10-09T13:23:13.861Z]   21: std::panic::catch_unwind
[2019-10-09T13:23:13.861Z]              at /rustc/c6e9c76c59e3c10acd63ca9ec157a8894ea1a068/src/libstd/panic.rs:394
[2019-10-09T13:23:13.861Z]   22: test::run_test::run_test_inner::{{closure}}
[2019-10-09T13:23:13.861Z]              at src/libtest/lib.rs:1413

https://internal.pingcap.net/idc-jenkins/blue/rest/organizations/jenkins/pipelines/tikv_continue_test/runs/88/nodes/20/steps/32/log/?start=0


go get error

following the instructions to add the package to the project found this issue.
Cleaned the mod cache, but the same result.

Hi @Accefy  we solved the issue! Could you install the latest version of permify-go (0.3.1) and test it. You can try with this command:
go get<EMAIL_ADDRESS>We can close this issue afterwards.

extra quick! thank you.

sorry, I close it prematurely:

Hi, @Accefy  can you try with this command go get github.com/Permify/permify-go@latest  after that if it doesn't work can you share the screen shot of the server.go file with us?

I saw that there is a new version (0.3.2) but the issue still remains.
Here is the full code:

After running go mod tidy command, does your go.mod file look like this?

This is the result:

Does it give the same error as before or has it changed?

Hello, I apologize for the late response, but I was trying to testing out correctly. The issue is solved, but I had to change the import path from:
github.com/Permify/permify-go
to:
github.com/Permify/permify-go/v1
I recommend to change this line in the readme.

Hi @Accefy, we'll update the read.me file accordingly.

Update readme please, still
import permify "github.com/Permify/permify-go"


index.html: Add warning about expression risks

This should work as a caveat as discussed in #26

Codecov Report

Merging #38 (99e6f77) into master (b608e39) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #38   +/-   ##
=========================================
  Coverage     69.17%   69.17%
  Complexity     3122     3122
=========================================
  Files           152      152
  Lines          9593     9593
  Branches       2008     2008
=========================================
  Hits           6636     6636
  Misses         2162     2162
  Partials        795      795

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

@ecki
After discussing an alternative version of the text with Arnout, we've gone with the version in git master.
TY!


Switching to shooting range issue

Whenever I type the .range command as a spectator, is says in the console a huge error. I am not sure why.

Can you show me what it looks like

Here.

try the new build i released and see if it has the same issue. (if it has a different message send that too)

Still doing the same thing.
Duplicate of #

Do you know what version of exiled you are currently using?

When a new round starts, this is what comes out of the logs. (It comes out red)
Exiled.Loader - Version 2.14.0
Duplicate of #

Sorry this plugin only works with version 3.0 of exiled, which is for the public beta of the 11.0 version of sl. If you want to update your server to the beta(which i recommend, it has cool stuff), the official sl discord has a server-faq channel that tells you how. (if you have any other issues you can make a new thread)

Okay, thanks.
On Sat, Sep 25, 2021 at 1:29 AM rayzer @.***> wrote:

Sorry this plugin only works with version 3.0 of exiled, which is for the
public beta of the 11.0 version of sl. If you want to update your server to
the beta(which i recommend, it has cool stuff), the official sl discord has
a server-faq channel that tells you how. (if you have any other issues you
can make a new thread)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rayzerbrain/Spectator-Shooting-Range/issues/1#issuecomment-927014040,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AVYP33U537RQO2JU7XMHCK3UDVM4HANCNFSM5EWYH4KQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.