[Pytorch] Swiglu implementation not aligned with jiterator version in probability

Hello there, sorry to bother again, it's during my investigation to issue #709, that I found some diff for implementation of swiglu been occur when the hidden_size become larger, here is a UT to reproduce:
import torch
import transformer_engine
from transformer_engine.pytorch import cpp_extensions as tex
from transformer_engine.pytorch.constants import TE_DType

# Adjust for different input
d1 = 256
d2 = 512

swiglu_fwd = """
template <typename T> T swiglu_fwd(T x, T y) {
    return float(x) * float(y) / (1.0f + ::exp(-float(x)));
}
"""
swiglu_bwd = """
template <typename T> T swiglu_bwd(T x, T y, T g, T& dx, T& dy) {
    float x_sigmoid = 1.0f / (1.0f + ::exp(-float(x)));
    dx = x_sigmoid * (1 + float(x) * (1.0f - x_sigmoid)) * float(g) * float(y);
    dy = float(x) * x_sigmoid * float(g);
}
"""
swiglu_fwd = torch.cuda.jiterator._create_jit_fn(swiglu_fwd)
swiglu_bwd = torch.cuda.jiterator._create_multi_output_jit_fn(swiglu_bwd, num_outputs=2)

class MySwiglu(torch.autograd.Function):
    @staticmethod
    def forward(ctx, inputmat):
        x1, x2 = torch.chunk(inputmat, 2, dim=-1)
        ctx.save_for_backward(x1, x2)
        return swiglu_fwd(x1, x2)

    @staticmethod
    def backward(ctx, dout):
        x1, x2 = ctx.saved_tensors
        return swiglu_bwd(x1, x2, dout)

myswiglu = MySwiglu.apply
input_mat1 = torch.rand(d2, d1).bfloat16().to('cuda')
input_mat2 = input_mat1.clone().detach().bfloat16().to('cuda')

r1 = myswiglu(input_mat1)
r2 = tex.swiglu(input_mat2, None, tex.FP8FwdTensors.GEMM2_INPUT, otype=TE_DType[torch.bfloat16])
print(r1)
print(r2)
print((r1-r2).nonzero())

So the thing is, I implemented a MySwiglu class, with fwd and bwd been compiled by torch.cuda.jiterator, and compared to tex version swiglu. Interesting thing is that, if you set L6 to d1 = 128, the final line diff almost didn't occur at all, while if d1 set to 256, there are some probability that some line of output is having mis-match, and when you set it to larger number for d1, let's say 1024 or 2048, diff result line appears more ...
I am not very sure if this is due to implementation bug of the jiterator's swiglu fwd, or it's something interesting inside the tex version swiglu ...

Some more setup information: torch version 2.1.0+cu122, TE version 1.3.0 python 3.9

Thanks @tylaar for reporting that. Let me take a look at that.

I did a little bit of digging and (at least looking at the case I tried) it does not seem to be a bug, but rather an artifact of the finite precision of the computations.
I modified your script a little bit to have deterministic execution and better show what happens:
import torch
import transformer_engine
from transformer_engine.pytorch import cpp_extensions as tex
from transformer_engine.pytorch.constants import TE_DType

torch.manual_seed(1234)
torch.set_printoptions(precision=10)

# Adjust for different input
d1 = 256
d2 = 512

swiglu_fwd = """
template <typename T> T swiglu_fwd(T x, T y) {
    return float(x) * float(y) / (1.0f + ::exp(-float(x)));
}
"""
swiglu_bwd = """
template <typename T> T swiglu_bwd(T x, T y, T g, T& dx, T& dy) {
    float x_sigmoid = 1.0f / (1.0f + ::exp(-float(x)));
    dx = x_sigmoid * (1 + float(x) * (1.0f - x_sigmoid)) * float(g) * float(y);
    dy = float(x) * x_sigmoid * float(g);
}
"""
swiglu_fwd = torch.cuda.jiterator._create_jit_fn(swiglu_fwd)
swiglu_bwd = torch.cuda.jiterator._create_multi_output_jit_fn(swiglu_bwd, num_outputs=2)

class MySwiglu(torch.autograd.Function):
    @staticmethod
    def forward(ctx, inputmat):
        x1, x2 = torch.chunk(inputmat, 2, dim=-1)
        ctx.save_for_backward(x1, x2)
        return swiglu_fwd(x1, x2)

    @staticmethod
    def backward(ctx, dout):
        x1, x2 = ctx.saved_tensors
        return swiglu_bwd(x1, x2, dout)

myswiglu = MySwiglu.apply
input_mat1 = torch.rand(d2, d1).bfloat16().to('cuda')
input_mat2 = input_mat1.clone().detach().bfloat16().to('cuda')

r1 = myswiglu(input_mat1)
r2 = tex.swiglu(input_mat2, None, tex.FP8FwdTensors.GEMM2_INPUT, otype=TE_DType[torch.bfloat16])
print(r1)
print(r2)
mismatches = (r1-r2).nonzero()
print(mismatches)
for m in mismatches:
    index = tuple(m)
    print(r1[index])
    print(r2[index])
    in1, in2 = torch.chunk(input_mat1, 2, dim=-1)
    x = in1.float()[index]
    y = in2.float()[index]
    out = x * y / (1 + torch.exp(-x))
    print(out)
    print(out.bfloat16())
    temp1 = 1 / (1 + torch.exp(-x))
    temp = x * temp1
    out2 = temp * y
    print(out2)
    print(out2.bfloat16())

I tried it on H100, CUDA 12.4 and TE 1.4 (we did not change the logic of swiglu in 1.4 compared with 1.3 so it should be completely equivalent).
 While it looks like a large difference, it is actually expected - bfloat16 has small number of mantissa bits and it turns out that those 2 values are just 2 consecutive values in bfloat16 range.
Now, why we see the difference is because the FP32 value, which is the "real" result of the swiglu computation, lies very close to the boundary between these 2 bfloat16 values, so small numerical differences may result in rounding in either direction. This is shown here:
    in1, in2 = torch.chunk(input_mat1, 2, dim=-1)
    x = in1.float()[index]
    y = in2.float()[index]
    out = x * y / (1 + torch.exp(-x))
    print(out)
    print(out.bfloat16())
    temp1 = 1 / (1 + torch.exp(-x))
    temp = x * temp1
    out2 = temp * y
    print(out2)
    print(out2.bfloat16())

which produces
tensor(0.4697265923, device='cuda:0')
tensor(0.4707031250, device='cuda:0', dtype=torch.bfloat16)
tensor(0.4697265625, device='cuda:0')
tensor(0.4687500000, device='cuda:0', dtype=torch.bfloat16)

out is computed according to the formula you used in swiglu_fwd function. As you can see, it is 0.4697265923, which is 0.0009765327 away from 0.4707031250 and 0.0009765923 away from 0.4687500000. Note that those 2 differences are very close to each other, but ultimately this value is closer to the bf16 value produced by your function. out2 is computed according to the equivalent formula, but which does computation in a slightly different order (which corresponds to what we internally do in the swiglu implementation). You can see that this result in FP32 is very slightly different (0.4697265625), which in this case results in rounding going in the other direction.

Thanks @ptrendx for your dedicated investigation and reply! That makes sense to me!
Just adding a little which could be interesting on the compiler level, I did a little change on my forked TE on the implementation of swish part inside math.h, by letting swish not calling sigmoid template, instead making it looks like:
template <typename OType, typename IType>
__device__ inline OType swish(const IType val, const Empty& e) {
    const float cval = val;
    return cval * 1.f / (1.f + exp(-cval));  // Which expand the sigmoid manually here
}

The forward diff will shrink down into almost 0 probability by iterating thousands of times in d1 = 1024 and d2 = 2048
I guess it's c++ level template type casting diff which causing the diff, but I haven't look at PTX level yet. However since you've explained  quite thoroughly, I think this is not going to be a issue for me then. Thanks a lot!


Update modules.ddoc

Hmm - why do you want to hide the API as long as ndslice is still in Mir?

Hmm - why do you want to hide the API as long as ndslice is still in Mir?

Documentation is much better in Phobos
Except sparse no other significant change will occur
We need to change macros in ndslice to support coorectly all links both in Mir and Phobos

Okay I see your point with #148 - feel free to merge.


DOC: Add docs for gcc/clang build

#36

@rgommers Please have a look at the added docs. Once, this looks I will open a PR in main repo.

Hi @Smit-create, here are some suggested changes

Thanks! I updated it.


Shell syntax

If we're going to make this a "photo shell" we should probably design that way from the begining. The first feature implemented is importing, so maybe a syntax for an import command would be useful?
Since it currently doesn't take any options, to start it could be as simple as:
import dir

Another good command (already implemented in the UI) would be next/previous. This might be part of the general select command (eg. select next and select previous which would later be expanded to allow selecting of multiple images, etc. once that functionality is in the UI).

Another possibility I thought of today: Maybe instead of creating our own simple syntax for photo processing, we just expose our internal API via a python REPL? Open it up in the app (or start in headless mode) and it automatically runs a file that imports everything you need from photoshell.


Adaptation for Thread 1.3

The following west manifest projects have been modified in this Pull Request:

Name
Old Revision
New Revision
Diff

openthread
https://github.com/zephyrproject-rtos/openthread/commit/e2a765599aa0379d0f72f0149a9cbafec070c12d
https://github.com/zephyrproject-rtos/openthread/commit/b21e99b4b3d823f71c902b9174ff62b964c124f0 (zephyr)
<EMAIL_ADDRESS>

Note: This message is automatically posted and updated by the Manifest GitHub Action.


README.md / filtering examples

The filtering capabilities of mbed-trace are not covered in the README.md
at all currently, a few examples how they could be used.

Some CI problems;
+ mbed deploy
fatal: reference is not a tree: e4764e89aab1fefc13ee052a7f5844b196e94604
[mbed] ERROR: Unable to update "mbed-trace" to rev #e4764e89aab1

Squashed a few commits and force pushed. @RomanSaveljev  - who has merge rights to this repo?

Quite a few people, including me. I can merge this especially as this really doesn't need CI.

Does the CI have an issue with getting stuff from forks? THat would explain why it can't find the hash, I think.

Thanks!

May well be. It's been broken for ages now though - no one really owns this repo so no one is motivated to fix it.


[4.0-pre] test failures on Windows

Running locally on Windows I have 14 failures in total.
7 of which is with specifying "hash" object as input sources to minify(), which the following seems to workaround them:
diff --git a/lib/reader/read-sources.js b/lib/reader/read-sources.js
index 1ccbdb2..9c3c2c3 100644
--- a/lib/reader/read-sources.js
+++ b/lib/reader/read-sources.js
@@ -57,7 +57,8 @@ function fromString(input, context, callback) {

 function fromArray(input, context, callback) {
   var inputAsImports = input.reduce(function (accumulator, uri) {
-    accumulator.push(restoreAsRelativeImport(uri));
+    uri = restoreAsRelativeImport(uri);
+    accumulator.push(restoreImport(uri, '') + Marker.SEMICOLON);
     return accumulator;
   }, []);

@@ -71,7 +72,8 @@ function fromHash(input, context, callback) {

   for (uri in input) {
     source = input[uri];
-    inputAsImports.push(restoreAsRelativeImport(uri));
+    uri = restoreAsRelativeImport(uri);
+    inputAsImports.push(restoreImport(uri, '') + Marker.SEMICOLON);

     context.sourcesContent[uri] = source.styles;

@@ -97,17 +99,14 @@ function trackSourceMap(sourceMap, uri, context) {
 function restoreAsRelativeImport(uri) {
   var currentPath = path.resolve('');
   var absoluteUri;
-  var relativeToCurrentPath;

   if (isRemoteResource(uri)) {
-    return restoreImport(uri, '') + Marker.SEMICOLON;
+    return uri;
   } else {
     absoluteUri = isAbsoluteResource(uri) ?
       uri :
       path.resolve(uri);
-    relativeToCurrentPath = path.relative(currentPath, absoluteUri);
-
-    return restoreImport(relativeToCurrentPath, '') + Marker.SEMICOLON;
+    return path.relative(currentPath, absoluteUri);
   }
 }

Thanks for looking into this. It needs to be green in time of 4.0, but, honestly, I neglected testing it as I go on Windows so far.

@jakubpawlowicz I suspected as much, so I thought I could look into this quickly without getting in the way 😛
Of the remaining 7 tests, 5 are disagreeing between \n and \r\n with input from strings or fixtures which is \n only. The last two are \ vs `/' in source map paths.
I'm not sure what the expectations are in those cases, so I'm afraid the first patch is all I can do for now...

It was actually all due to paths being stored in inconsistent form. Fixed in 909460778f78b7ece96fd2e040cec70b035331e4.
I'll make sure to keep master green on both Windows and *nix from now on.

So I'm still getting 5 test failures on Windows 10 and Node.js 7.3.0
C:\...\clean-css>npm test

><EMAIL_ADDRESS>test C:\...\clean-css
> vows

 ······························································································································································································································································································································································································································································································································································································ ·················································································· · ············································································································································································································································· ··················································································································································· ······················································································································································ ·············································································································································································································································································································································································································································································································································································································· ········································· ······················································································· ············ ························ ···················· ···························· ························ ···························· ··························· ······ ···················· ······························ ····· ······· ············ ······ ······································ ······································ ········································· ······· ·· ··········· ·············································································· ···························································· ········· ·································································································································································✗··························✗······✗···✗··✗······················ ······················································································ ······ ·········· ··········· ·······················

    inlined sources from array - on
      ✗ has embedded sources content
        » expected [ '.one { color:#f00; }\015\n', '.three {background-image: url(extra/down.gif);}\015\n' ],
        got      [ '.one { color:#f00; }\n', '.three {background-image: url(extra/down.gif);}\n' ] (deepEqual) // C:\...\clean-css\test\source-map-test.js:1360

    inlined sources from source map(s) mixed
      ✗ has embedded sources content
        » expected [ 'div {\n  color: red;\n}\n', 'section {\n  > div a {\n    color:red;\n  }\n}\n', 'div > a {\015\n  color: blue;\015\n}\015\n' ],
        got      [ 'div {\n  color: red;\n}\n', 'section {\n  > div a {\n    color:red;\n  }\n}\n', 'div > a {\n  color: blue;\n}\n' ] (deepEqual) // C:\...\clean-css\test\source-map-test.js:1641

    inlined sources from source map(s) mixed remote and no callback
      ✗ has embedded sources content
        » expected [ null, 'section {\015\n  > div a {\015\n    color:red;\015\n  }\015\n}\015\n', null ],
        got      [ null, 'section {\n  > div a {\n    color:red;\n  }\n}\n', null ] (deepEqual) // C:\...\clean-css\test\source-map-test.js:1806

    inlined sources from source map(s) mixed remote and 404 resource
      ✗ has embedded sources content
        » expected [ null, 'section {\015\n  > div a {\015\n    color:red;\015\n  }\015\n}\015\n', 'div > a {\n  color: blue;\n}\n' ],
        got      [ null, 'section {\n  > div a {\n    color:red;\n  }\n}\n', 'div > a {\n  color: blue;\n}\n' ] (deepEqual) // C:\...\clean-css\test\source-map-test.js:1761

    inlined sources from source map(s) mixed remote
      ✗ has embedded sources content
        » expected [ 'div {\n  color: red;\n}\n', 'section {\015\n  > div a {\015\n    color:red;\015\n  }\015\n}\015\n', 'div > a {\n  color: blue;\n}\n' ],
        got      [ 'div {\n  color: red;\n}\n', 'section {\n  > div a {\n    color:red;\n  }\n}\n', 'div > a {\n  color: blue;\n}\n' ] (deepEqual) // C:\...\clean-css\test\source-map-test.js:1713
  ✗ Broken » 2798 honored ∙ 5 broken (16.798s)
  npm ERR! Test failed.  See above for more details.

All of them is caused by this line, where it is defined as \r\n whereas the actual output were all \n.

That's interesting as it works ok on Node.js 4 and 6 - https://ci.appveyor.com/project/jakubpawlowicz/clean-css
I can give it a try on 7 tomorrow.

I would try those too except all my Windows boxes have Node 7 on them. :sweat:
It would be helpful to have CI tests for Node 7 on Linux and Windows anyway.
One possibility may be that git clone is "fixing up" line endings of those fixtures files, which I always disable (leave them as-is).

Could be the thing about git clone, see https://ci.appveyor.com/project/jakubpawlowicz/clean-css/build/690 and https://travis-ci.org/jakubpawlowicz/clean-css/builds/190870717.
Btw, shouldn't we officially support Node.js 4, 6, and 7, given the last one is the current one?

@jakubpawlowicz apologies for being unclear - I do mean v7 in addition to v4 & v6.

Yeah, just a git clone artefact which won't affect any real world use cases then. :+1:

Cool! Let's target for v4, v6, and v8 in clean-css 4.


Fix thread safety in http pool manager

Problem
There is a thread safety issue in the way the HttpClient initializes. Here is a snippet to trigger it:
import multiprocessing

import clickhouse_connect

def get_client() -> clickhouse_connect.driver.client.Client:
    return clickhouse_connect.get_client(host='localhost')

def test_client(*_):
    client = get_client()
    client.command('SELECT version();')
    client.close()

def test_concurrency():
    # Call test client in original thread, before creating new
    # process. This ensures that any globals that may be initialized
    # in the library are initialized before forking next
    test_client()

    # Fork 8 processes, each initializing a fresh client. Any globals
    # from this process that end up being used in child processes may
    # very well become broken.
    with multiprocessing.Pool(8) as pool:
        pool.map(test_client, range(100))

if __name__ == '__main__':
    test_concurrency()

We initialize a client in the parent process, make a request for the version and close the client
We fork 8 times, and run the same version request all with fresh clients that are closed at the end

This gives us this lovely error:
Traceback (most recent call last):
  File "/home/jc/vola/git/experimental/data1backup/tests/clickhouse-connect/clickhouse_connect/driver/httpclient.py", line 322, in _raw_request
    response: HTTPResponse = self.http.request(method, url,
  File "/opt/conda/envs/py310_64/lib/python3.10/site-packages/urllib3/request.py", line 78, in request
    return self.request_encode_body(
  File "/opt/conda/envs/py310_64/lib/python3.10/site-packages/urllib3/request.py", line 170, in request_encode_body
    return self.urlopen(method, url, **extra_kw)
  File "/opt/conda/envs/py310_64/lib/python3.10/site-packages/urllib3/poolmanager.py", line 376, in urlopen
    response = conn.urlopen(method, u.request_uri, **kw)
  File "/opt/conda/envs/py310_64/lib/python3.10/site-packages/urllib3/connectionpool.py", line 785, in urlopen
    retries = retries.increment(
  File "/opt/conda/envs/py310_64/lib/python3.10/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/opt/conda/envs/py310_64/lib/python3.10/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/opt/conda/envs/py310_64/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/opt/conda/envs/py310_64/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/opt/conda/envs/py310_64/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/opt/conda/envs/py310_64/lib/python3.10/http/client.py", line 1374, in getresponse
    response.begin()
  File "/opt/conda/envs/py310_64/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/opt/conda/envs/py310_64/lib/python3.10/http/client.py", line 300, in _read_status
    raise BadStatusLine(line)
urllib3.exceptions.ProtocolError: ('Connection aborted.', BadStatusLine('\r\n'))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/conda/envs/py310_64/lib/python3.10/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/opt/conda/envs/py310_64/lib/python3.10/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "/home/jc/vola/git/experimental/data1backup/tests/test-concurrency.py", line 11, in test_client
    client = get_client()
  File "/home/jc/vola/git/experimental/data1backup/tests/test-concurrency.py", line 7, in get_client
    return clickhouse_connect.get_client(host='localhost')
  File "/home/jc/vola/git/experimental/data1backup/tests/clickhouse-connect/clickhouse_connect/__init__.py", line 8, in get_client
    return create_client(**kwargs)
  File "/home/jc/vola/git/experimental/data1backup/tests/clickhouse-connect/clickhouse_connect/driver/__init__.py", line 69, in create_client
    return HttpClient(interface, host, port, username, password, database, settings=settings, **kwargs)
  File "/home/jc/vola/git/experimental/data1backup/tests/clickhouse-connect/clickhouse_connect/driver/httpclient.py", line 150, in __init__
    super().__init__(database=database, query_limit=coerce_int(query_limit), uri=self.url)
  File "/home/jc/vola/git/experimental/data1backup/tests/clickhouse-connect/clickhouse_connect/driver/client.py", line 45, in __init__
    tuple(self.command('SELECT version(), timezone(), database()', use_database=False))
  File "/home/jc/vola/git/experimental/data1backup/tests/clickhouse-connect/clickhouse_connect/driver/httpclient.py", line 288, in command
    response = self._raw_request(payload, params, headers, method)
  File "/home/jc/vola/git/experimental/data1backup/tests/clickhouse-connect/clickhouse_connect/driver/httpclient.py", line 341, in _raw_request
    raise OperationalError(f'Error executing HTTP request {self.url}') from ex
clickhouse_connect.driver.exceptions.OperationalError: Error executing HTTP request http://localhost:8123

The problem is the global clickhouse_connect.driver.httputil.default_pool_manager, which is used only as the default http pool in clickhouse_connect.driver.httpclient.HttpClient.init
When this global is initialized in one process, then referenced in another it completely breaks any attempt at concurrency.
Solution
Remove the clickhouse_connect.driver.httputil.default_pool_manager global, and initialize a new pool manager on each new HttpClient.
Unless there is some huge perfomance cost to initializing new pool managers that I am unaware of, this should be the way it is done.

I have identified another issue with thread safety. The default session_id is generated from uuid.uuid1 which is not generated in a multiprcessing safe way. This means that if two threads call the function at very near times, you can get the same uuid. This breaks the session locking stuff server side. Due to the nature of this, it can be a bit annoying to trigger.
I have switched the uuid.uuid1 to uuid.uuid4, which has significantly higher entropy.

Thanks for your investigations into multiprocessing! It's an area that we simply haven't had the time to test the limits.
I'm very curious about the error you see when using the default_pool_manager.  It's my understanding the urllib3 pool manager is thread safe and even before that PR the window where it was not thread safe was tiny (and would raise a different exception).  In particular I'm fairly sure the actual connections are never shared between threads, although it's possible that the actual Python http_client somehow gets shared.  Is it possible that the "bad status" line is just the result of some other ClickHouse server or  loopback issue, possibly related to the load of 100 clients hitting the same ClickHouse server at once?
In any case the choice to use a single PoolManager by default was intended for the simple and I assume most common use case of having a few clients always hitting the same host.  Reusing connections in that case is desirable, especially if they can reuse "keep alive" HTTP connections.  (However, looking through the code I did notice that the original maxsize of the pool was lost when removing the requests library - it was 8, unfortunately now I think it's defaulting to 1 - so that might be worth fixing.)
In the case of a large number of clients/threads my intended solution was to construct those clients with their own PoolManager, preferable obtained by the get_pool_manager method.  I consider that an advanced use case but it should fix your particular issue without a code change.  Just change this line:
client = get_client()

to
client = get_client(pool_mgr=get_pool_manager())

You could also use that mechanism to share a few different pool managers among multiple hosts, instead of having a 1 to 1 assignment.
As for uuid1 vs uuid4, there's still 14 bits of random entropy on a single host in the same timestamp, vs 64 bits of entropy across all possible clients hitting the same ClickHouse server.  I think there's no practical difference, although I can see uuid4 as being marginally "safer".   But this is also something you can solve without a code change by settings the session_id parameter in the get_client factory method call.

btw the way your concurrency code snippit does not trigger any exceptions on my M1 Mac -- Python 3.9.15, urllib 1.26.13

It is occuring every single run on arch linux with my 3950x (16 cores 32 threads). I also confirmed this issue on my XPS 9710 running Manjaro linux.
If you are running your python through the hacky emulation layer thing they made for M1s, I suspect that python multiprocessing in general may be handled very differently.

It's native ARM64, not Rosetta, but again that doesn't actually look like a concurrency error.

As it turns out I was sort of right in my assumption about macOS doing multiprocessing differently:
https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods

Changed in version 3.8: On macOS, the spawn start method is now the default. The fork start method should be considered unsafe as it can lead to crashes of the subprocess. See bpo-33725.

Updating the my example:
import multiprocessing

import clickhouse_connect

def get_client() -> clickhouse_connect.driver.client.Client:
    return clickhouse_connect.get_client(host='localhost')

def test_client(*_):
    client = get_client()
    client.command('SELECT version();')
    client.close()

def test_concurrency():
    # Force non-default spwan start method
    multiprocessing.set_start_method('spawn')

    # Call test client in original thread, before creating new
    # process. This ensures that any globals that may be initialized
    # in the library are initialized before forking next
    test_client()

    # Fork 8 processes, each initializing a fresh client. Any globals
    # from this process that end up being used in child processes may
    # very well become broken.
    with multiprocessing.Pool(8) as pool:
        pool.map(test_client, range(100))

if __name__ == '__main__':
    test_concurrency()

I would not consider doing this multiprocessing.set_start_method as an acceptable solution.
If this library is being developed only for macOS, then you need to disclose this up front and in the documentation. My org is simply not going to use clickhouse if this is the case.

We build and test on linux, but I happen to develop on a Mac.  And as I alluded to above, we don't currently have automated tests for large parallel processing.
ClickHouse Connect is still very much beta and open source, so you know everything there is to know about tests and builds.  Again, the urllib3 PoolManager is considered thread safe, but it appears you have found an exception.  If you can reproduce the problem independent of ClickHouse Connect (which shouldn't be difficult, it really amounts to a bunch of parallel HTTP calls) I would suggest you open an issue in that project.
It also seems like if you're going to use multiprocessing on Linux/Unix, you should not use a start method that is "problematic" according to the documentation and then blame the library.
In any case, there's a simple workaround for the issue as I described above for your use case that doesn't require a code change and doesn't disable the current default connection sharing, and you have identified yet another workaround.  As we have time and resources to update the documentation and move toward production status, those things will be improved.  In the meantime, you are of course free to fork this project, use another Python client (clickhouse-driver is well established) or use other database software entirely.

For the record, the python documentation is clear in saying that the "problematic" fork start method is only problematic on macOS. It is still the standard on linux because linux can handle posix forking.

https://bugs.python.org/issue40379 -- not just MacOS


Resolver definitions from CAS

Hello,
I am currently using shib-cas-authn3 to successfully authenticate Shib through my CAS server. I am trying to write my attribute-resolver.xml file for use with a vendor. I need to access the user fields defined in my CAS deployerConfigContext.xml. However I can only figure out how to get the PrincipalName. Is it possible to see all the information provided by CAS in a resolver? Or do I have to take the PrincipalName and search LDAP for the fields I need?

This is presently not possible. You can only get back from the principal from CAS. Attributes that are going back to the SP must be resolved and released by the idp.

Okay, thank you for the information. I will use LDAP to pull the information I need.


Added Dash

Added Dash with SHA3 crypto for X11 header hash calculation, Mainnet works fine.

Awesome! Can you add it to NBitcoin.TestFramework ? The "altcoins" tests are doing some very basic additional checks on regtest.

Ah you did! wonderful

System.Exception : Hash of downloaded file does not match (Expected: 19768b5395ce072387a2c130b65f4fdd6d7bf16a7cef88c0acfc46bd759201e2, Actual: 04e95d11443d785ad9d98b04fd2313ca96d937e424be80f639b73846304d154c)
They changed the file ?

Most likely I copied wrong hash, I tested only mainnet with NBXplorer, will push that part tomorrow. Alao I’m not sure about PoW, will look into tomorrow too.

That's fine doing now! awesome job the tests pass

Pushed new version of NBitcoin.Altcoins and NBitcoin.TestFramework.

Awesome, Thanks! I have one more coin coming soon, BitcoinGold. ConsensusFactory is incredible.

@bdangh this looks awesome, great job!


Yolov5s conversion works with rknn_toolkit2, but not with default ONNX export parameters

Using the yolov5s.onnx supplied in the toolkit demo conversion results in a working rknn model, but using default
export.py from ultralytics results in the one layer model that selects all possible bounding boxes.

I am currently struggling with this issue as well. @neocoretechs did you find a work around?

Update: Khadas forked the ultralytics/yolov5 repo and that script worked. https://github.com/rockchip-linux/rknpu2/issues/57


Record list does not update after edit

Let's consider :
Parent module with one field : "filterField" (string)
Child module with one field :  "name" (string)
The record of parent module has two blocks :

one record block with just filterField
one record list block for child module records with a filter : name = '${record.values.filterField}'

Expected behavior :
When changing and saving filterField --> the record list filter will change
Actual behavior :
Nothing changes, until the page is reloaded (F5).

The correction of this issue 1672 might correct this one as well

Yup as you pointed out, the #1672 issue is for this exact thing. Closing this one so we can keep it all in one


more reliable algorithms for choosing which questions to post

I don't know if this matters at all. I suspect probably not. But it was an interesting puzzle that I got nerd-sniped by, so I figured I'd right down what I thought of :-)
The bot's goal is to post questions without any duplicates or missing any, ie exactly-once delivery. As we know from distributed systems theory, reliable exactly-once delivery is ludicrously difficult or impossible, so we want to pick some "good enough" heuristic that isn't too complicated to implement. So the challenge is to optimize that reliability/simplicity tradeoff.
Right now the bot uses an extremely simple heuristic: it polls for questions every ten minutes, and then it posts any questions that have timestamps newer than ten minutes. I love this because if you think of it from a distributed systems perspective it's obviously too simple to work (schedulers aren't accurate! We're comparing clocks across two completely separate platforms!), but in fact it (probably) works very well and it's so simple that it's actually hard to beat.
It does at least theoretically have flaws though: successive ten minute windows are never going to line up exactly; there will always be a gap or some overlap. I guess probably a few seconds every ten minutes. If a question happens to be posted during that time, then it will be either double-posted or lost entirely. Can this be avoided, without massively complicating the implementation?
I did have one clever idea: when we start up, fetch the feed and store a list in-memory of which questions are already there. Then on future iterations, fetch the list, and post all the questions that aren't in our in-memory list, and add them to the list. That totally removes the dependency on accurate clocks, and is still super simple. It does have the downside that if a question is posted a few minutes before the bot is restarted, it might get lost entirely. I guess we could try to minimize that chance by using a hybrid of the two approaches: schedule our regular checks at a known absolute time (like: not just every ten minutes, but at 1:00, 1:10, 1:20, etc.), and then at startup (only) use the question timestamps and assume that anything since the last scheduled check time was probably not posted by our previous incarnation, so we should post it.
Or, the other obvious approach would be to keep a record of what questions have already been posted. Heroku makes it very easy to attach a database to a project, and at the scale we need it's free (less than 10k rows). This does add moderate complexity though, since we have to pick a postgres client (async or sync?), set up a simple schema, remind ourselves how to do SQL, and at least think about garbage collection. It's still pretty simple though, and TBH it might score better overall than my Very Clever solution. Sigh.

@jtrakk made an interesting suggestion on gitter:

I wonder if the bot could just ask SO for this week's trio questions, and ask gitter for this week's posts by trio-bot, and post any missing ones?

It doesn't look like there's any way to list the bot's posts specifically, but one could do something like, iterate over messages until you find the last thing the bot said or have gone an hour back in time.
Still dubious that it's worth actually worth the effort to implement, but it's a nice approach! Probably nicer than any of the other options mentioned above.

If we're open to commercial solution, we can use Zapier to detect the new item in RSS, which has pretty sophisticated way to detect which item is new or not (Note: I work at Zapier, so I have insider knowlege) https://zapier.com/apps/rss/integrations
So instead of maintaining our own database, writing code for detecting new item, etc, we use Zapier. We can even build our own gitter Zapier integration.


Not working in IL2CPP

Symptoms
Extremely slow audio. There are also too many memory allocations, possible bug during MeltySynth conversion to cpp.

:tada: This issue has been resolved in version 1.10.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

It's very nice to know that MeltySynth works on IL2CPP 😄
Was the bad thing for IL2CPP the Vector<float> version of MultiplyAdd?

Hey @sinshu, tks for watch this issue!
Yes! After long time profiling an android device, i see MemoryMarshal.Cast and Vector<float> inside MultiplyAdd using 100% of cpu (Audio Thread) and too many GC alloc.

This solve the CPU Usage (main problem), but i still see lot of GC alloc in some parts. I will attach the profile results here later.


The problem is in network with NAT

INTERNET
  |
  |
  Q [<IP_ADDRESS>, <IP_ADDRESS>] - Qora node 1
   \
    \
     Q [<IP_ADDRESS>] - Qora node 2

At the same time the nodes do not work or work unstable.

https://github.com/Qoracoin/Qora/commit/f2340a738d743b77fe9943f9c7a3f8c0c55c55ce


Tested drive Samsung 980 Pro

:tada: This PR is included in version 0.8.21 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


added kroxylicious scale up and down test cases

Type of change

Enhancement / new feature

Description
Added two new system test for scaling up and down the kroxylicious instances.
Additional Context
Fixes #988
Checklist

[X] Write tests
[X] Make sure all tests pass
[ ] Review performance test results. Ensure that any degradations to performance numbers are understood and justified.
[ ] Make sure all Sonarcloud warnings are addressed or are justifiably ignored.
[ ] Update documentation
[ ] Reference relevant issue(s) and close them after merging
[ ] For user facing changes, update CHANGELOG.md (remember to include changes affecting the API of the test artefacts too).

@strimzi-ci run system tests

:arrow_forward: Build started - check Jenkins for more info. :arrow_forward:

:heavy_check_mark: Test Summary :heavy_check_mark:
BUILD NUMBER: 17
KAFKA VERSION: 3.6.1
TOTAL: 8
PASS: 7
FAIL: 0
SKIP: 1

do you know why the build is failing with import sorting?
Error:  Failed to execute goal net.revelc.code:impsort-maven-plugin:1.9.0:check (sort-imports) on project kroxylicious-systemtests: Imports are not sorted in /home/runner/work/kroxylicious/kroxylicious/kroxylicious-systemtests/src/main/java/io/kroxylicious/systemtests/installation/vault/Vault.java -> [Help 1]

On Intellij the imports are ordered automatically (I have this configuration) but it is the first time it is not doing the ordering as expected.


Roundtripping command outputs

The front page lists two commands:

exec > >(nc streamhut.io 1337) 2>&1
exec 3<>/dev/tcp/streamhut.io/1337 && head -1 <&3 && exec &> >(tee >(cat >&3))

However those work very differently, since the second one will only write to the socket (cat >&3) and duplicate the terminal output to the socket and terminal (tee), leaving streamhut's socket output on fd 3 (which doesn't go to the terminal), while the first one prints pipes all output through streamhut, relying on it printing back everything it gets.
This means that the first command:

introduces a noticeable delay
needs streamhut.io needs to send back everything it receives (wasting bandwidth)
it's technically possible for streamhut to subvert your command outputs (not just changing what's printed on the website, but changing what's printed to your terminal)

I'm not completely sure how to fix the command since exec 3<>(nc streamhut.io 1337) won't work :thinking:
Otherwise this app looks very nice! I was just surprised to see that those two commands, which are presented as equivalent (just the "no nc alternative") are in fact not (second is much better) and that the workings of the first one puts more demand on your infrastructure.

Something like this maybe?
exec 3> >(nc streamhut.io 1337 | head -1) && exec &> >(tee >(cat >&3))

It looks like streamhut is sending the first of my text back before the "streaming to ..." line though. But that's not a problem anymore once you turn off echo in the server.

You do make a good point about people understanding what they run -- my command definitely looks more contrived. Oh well, if nothing else it will be here if someone searches for it.


CHINA SOUTHERN B787-9(787th BOEING 787 in the world Livery）

@kolos26 When do you plan to fix the yellow line of Hainan Airlines and Hong Kong Airlines A359?

and is it possible for you to get a license for LRX?

do you konw free HD?forever

@kolos26 When do you plan to fix the yellow line of Hainan Airlines and Hong Kong Airlines A359?

Probably next week. I was busy 'cause of school.


Valibot to JSON Schema

Will there a native valibot ↔ json schema conversion?
I think this is what missing from zod (you have to use zod-to-json-schema, which is a third party package with some known issues)
Whereas in the python ecosystem, pydantic has native support to generate json schemas from models.
Lovely work so far!

1

but, I would also enjoy the other way around - json-schema to valibot schemas - perhaps we can leverage existing tools around zod/arktype for this...

It's on my list to check out a possible implementation. Since I'm currently writing my bachelor's thesis, I won't get to it until September.

Maybe https://github.com/sinclairzx81/typebox could be used as inspiration?

There is one existing solution to convert from valibot to json schema. You can generate json schema from the TS types with ts-json-schema-generator.
Of course it's not a perfect solution because some of the features of valibot do not translate to TS types but would translate to JSON schema (like minLength to constrain array size).
An example on how to use this:

define a schema

src/schema.ts

import { object, string, type Input } from 'valibot';

const FooElementSchema = object({ type: string('foo') });
export type FooElement = Input<typeof FooElementSchema>;

then execute npx ts-json-schema-generator --path src/schema.ts -t FooElement -o dist/schema.json

The result:
{
  "$ref": "#/definitions/FooElement",
  "$schema": "http://json-schema.org/draft-07/schema#",
  "definitions": {
    "FooElement": {
      "additionalProperties": false,
      "description": "Input inference type.",
      "properties": {
        "type": {
          "description": "Input inference type.",
          "type": "string"
        }
      },
      "required": [
        "type"
      ],
      "type": "object"
    }
  }
}

Thanks for the tip. Here is what I can currently say about this issue: We are working in PR #211 to make the properties of the schemas and validation functions accessible. This would theoretically allow direct serialization. However, it is important to note that serialization is not a goal of Valibot. This has to do with the fact that Valibot prioritizes functionality, bundle size and DX over serialization, since not every functionality can be serialized. Also, validating against a JSON schema as in TypeBox prevents tree shaking and code splitting. In the long run, however, I can imagine offering a JSON variant with the same API under valibot/json.

I've started to build myself a small CLI tool to convert from Valibot to JSON schema:
https://github.com/gcornut/valibot-json-schema
It loads a JS/TS module with esbuild-runner and manually converts the exported Valibot schemas into JSON schema. For my use case I don't mind having a big JS module doing the conversion of all schemas but I totally see how that would be a problem in a modular library like Valibot.
Like you say, some more metadata on schema validations (minLength, email, etc.) would help. In the meantime I might manually attach some custom metadata.
Something like:
import { withJSONSchemaFeatures } from '@gcornut/valibot-json-schema/extension';
const MyArray = withJSONSchemaFeatures(array([minLength(1)]), { minItems: 1 });
// => returns the ArraySchema with a custom Symbol keyed property used by internally during conversion to JSON schema

It's redundant but it's could be a cheap & easy way to do this 🤔

Thank you, thats great! I plan to merge PR #211 this week or next. If your library works reliably, feel free to add it to our ecosystem page.

Thank you, thats great! I plan to merge PR #211 this week or next. If your library works reliably, feel free to add it to our ecosystem page.

Ok my tool/library to convert to JSON schema is now pretty complete thanks to the new reflections API on pipe validation!
I opened a PR to add it to the ecosystem page https://github.com/fabian-hiller/valibot/pull/263

A native Valibot to JSON schema or JSON schema to Valibot implementation is not currently planned. Technically, however, it is largely possible through our new reflection API and can be covered by third-party packages like valibot-json-schema. If JSON Schema is important to you, I suggest you take a look at TypeBox.
I am therefore closing this issue. However, you can still contact me through this issue if you have any questions or feedback.

https://valibot.dev/guides/json-schema/


After gc, The number of pfree is getting smaller.

Hi, Boss:
Sometime, After gc, The number of pfree is getting smaller. and then need to try too many times. See below a piece of testsuite log .
And, How to setup the Garbage collecting heuristics when config with small page (64 Bytes)?
(block is 4KB)
Thanks a lot!
...
gc_check #33: run gc free_blocks:2 pfree:58 pallo:9285 pdele:22277 [31620] len:236 of 3422
gc_check: cleaning block 86, result 0

gc_check #34: run gc free_blocks:2 pfree:57 pallo:9285 pdele:22278 [31620] len:236 of 3363
gc_check: cleaning block 85, result 0

gc_check #35: run gc free_blocks:2 pfree:56 pallo:9285 pdele:22279 [31620] len:236 of 3304
gc_check: cleaning block 82, result 0

gc_check #36: run gc free_blocks:2 pfree:58 pallo:9285 pdele:22277 [31620] len:236 of 3422
gc_check: cleaning block 81, result 0

gc_check #37: run gc free_blocks:2 pfree:57 pallo:9285 pdele:22278 [31620] len:236 of 3363
gc_check: cleaning block 80, result 0

gc_check #38: run gc free_blocks:2 pfree:56 pallo:9285 pdele:22279 [31620] len:236 of 3304
gc_check: cleaning block 109, result 0

gc_check #39: run gc free_blocks:2 pfree:55 pallo:9285 pdele:22280 [31620] len:236 of 3245
gc_check: cleaning block 108, result 0

gc_check #40: run gc free_blocks:2 pfree:54 pallo:9285 pdele:22281 [31620] len:236 of 3186
gc_check: cleaning block 107, result 0

gc_check #41: run gc free_blocks:2 pfree:53 pallo:9285 pdele:22282 [31620] len:236 of 3127
gc_check: cleaning block 106, result 0

gc_check #42: run gc free_blocks:2 pfree:52 pallo:9285 pdele:22283 [31620] len:236 of 3068
gc_check: cleaning block 119, result 0
gc_check: finished, 31547 dirty, blocks 3 free, 73 pages free, 43 tries, res 0
...

Sorry for belated answer.
Yes, on a scattered filesystem this may happen - though it should fix itself after a while. The reason is that when a file data page is moved from a block being gc:ed, the index page must be updated. If this index page is another block than the one being gc:ed, the old one is deleted. However the assumption is that during a gc, index pages and belonging data pages ought to end up in the same block.
There is a fix for this which actually is under development now, but it requires more or less a full rewrite of the fs.
Generally, I would not recommend 64 byte logical page size but if that works for you go ahead. How many blocks do your filesystem have?

My norflash size is 2~4MB, and the physical erase block size is 4KB, which may split it into small pages and  logic blocks.  For examples:
16B * 256,
64B * 64,
128B * 32,
256B * 16,
...
I don't know which is the better one.
There has some small files and a few update operation in my filesystem.

Sorry for belated answer. I'd go with something like 128-256 byte page size, and perhaps 16-32-64k logical blocks. Reg. the GC heuristics, there probably is no global maximum. If it works fine for you now, I would not tinker with them. Otherwise, it is best to set up a usage model on your PC and use the testbench to simulate a year of usage and try to lock in what heuristics are optimal.
Unless you're doing lots of writes and removes, this might be unnecessary.


Can't change onedark style in init.lua.

I am using the kickstart.nvim script and I'm trying to change the style of the onedark theme being used. For example, to "light" instead of "dark".
To reproduce this bug,
In init.lua, inside the lazy setup, add the following line under line 139:
vim.g.onedark_config = { style = 'light' }

Desktop
OS: Mac OS 12.6.8
Terminal: iTerm2 3.4.20
Neovim Version
NVIM v0.9.1
Build type: Release
LuaJIT 2.1.0-beta

I figured out the problem. The style configuration needs to be provided before the color scheme is set. I assume this is because nvim does not read the updated configuration afterwards.


How about adding AI based solutions to maintain this repository?

SWE-agent: https://github.com/princeton-nlp/SWE-agent/issues/23
A good experience with CodeRabbit: https://github.com/pankona/pankona.github.com/pull/228#discussion_r1475470927
Bad experiences with CodeRabbit:

https://github.com/pankona/pankona.github.com/issues/204#issuecomment-1945368019
https://github.com/pankona/pankona.github.com/pull/304#discussion_r1733201252
https://github.com/pankona/pankona.github.com/pull/311#discussion_r1758782105

Neutral experience for Copilot https://github.com/kachick/dotfiles/pull/443#issuecomment-1983234729

Thump her and be bam bam

CodeRabbit looks like provided for response code checker 🙄

It is helpful to get boilerplate faster, however I couldn't get any good experience around dotfiles yet...

GitHub introduced beta feature...

https://github.com/orgs/community/discussions/139872

Looks not useful for now... 😢

🤷♂


Newsletter announcement spacing

@hevertonrodrigues told me the spacing was a little bit off, so I'm trying to follow his ideas

          1 Error

:no_entry_sign:
Found 2 link issues

          1 Warning

:warning:
Too many changes (when adding, please keep it to one project per Pull Request)

Link issues by awesome_bot

Line
Status
Link

178
301
http://www.codeproject.com/Articles/88929/Getting-Started-with-iPhone-and-iOS-Development redirects tohttps://www.codeproject.com/articles/88929/getting-started-with-iphone-and-ios-development

995
301
https://github.com/alvinvarghese/EasyIAPs redirects tohttps://github.com/aaalveee/EasyIAPs

  Generated by :no_entry_sign: danger

Uh this is a temporary change?

@dkhamsing it would be a temporary thing since we're working on the website, which will have an AwesomeiOS.Weekly page.
I understand that you didn't like the changes... Do you have any ideas on how to make it better?

I think it is fine if it is temporary :)


publish<EMAIL_ADDRESS>

Requested by: @kamilogorek
Merge target: (default)
Quick links:

View changes
View check runs

Assign the accepted label to this issue to approve the release.
Leave the #retract comment under this issue to retract the release (original issuer only).
Targets

[ ] gcs
[ ] npm
[ ] brew
[ ] github
[ ] registry

Targets marked with a checkbox have already been executed.  Administrators can manually tick a checkbox to force craft to skip it.

#retract

I fixed everything manually. GitHub was down, which made GitHub target fail and effectively get our releases out-of-sync.
There is no merge of 1.74.1 https://github.com/getsentry/sentry-cli/commits/master but its fine, as npm/brew and github artifacts were released correctly.


Initial Setup of non mvx plugin

[x] setup projects
[ ] move code to Common library

I see you are doing a lot of structural changes. I'll refrain from changing anything in these files so that we avoid conflicts. Let me know when you are ready to merge this branch back into master.

Moved the ios and android implementations to Plugin.Ble.*

Assignee: sms ^^ iwas working on this. we should talk. Where are you today? come to work.


MountVolume.MountDevice failed for volume "pvc-XXX": rpc error: code = Internal desc = Waiting for pvc-XXX CVC to be bound

Hey folks,
I have successfully installed and configured the OpenEBS cStor.
$ k -n openebs get pod
NAME                                                              READY   STATUS    RESTARTS      AGE
cstor-disk-pool-729z-65bfd9d85b-26nsj                             3/3     Running   0             16h
cstor-disk-pool-kq22-5866d99f9-zbcj4                              3/3     Running   0             39m
cstor-disk-pool-xr9s-5cf7444dd7-cr9tm                             3/3     Running   0             16h
openebs-cstor-admission-server-6cb66f495d-8pzxd                   1/1     Running   0             39m
openebs-cstor-csi-controller-0                                    6/6     Running   0             39m
openebs-cstor-csi-node-4lsz9                                      2/2     Running   2 (34m ago)   3d22h
openebs-cstor-csi-node-84khx                                      2/2     Running   0             3d22h
openebs-cstor-csi-node-r2gd9                                      2/2     Running   0             3d22h
openebs-cstor-cspc-operator-597bdbd5bb-gqb9w                      1/1     Running   0             39m
openebs-cstor-cvc-operator-565bcccf-wkjw9                         1/1     Running   0             39m
openebs-localpv-provisioner-7b4db8497-k9glx                       1/1     Running   0             39m
openebs-ndm-g8pl9                                                 1/1     Running   0             3d22h
openebs-ndm-operator-575c7c66c-ksqq7                              1/1     Running   0             39m
openebs-ndm-tqblt                                                 1/1     Running   2 (33m ago)   3d22h
openebs-ndm-zppb2                                                 1/1     Running   0             3d22h
pvc-f73ae1f8-66c9-4e4c-b7d7-44e2c80a2a93-target-7867cb87c-8cgt2   3/3     Running   0             3m58s

$ k -n openebs get bd
NAME                                           NODENAME   SIZE          CLAIMSTATE   STATUS   AGE
blockdevice-b9325aef61771319097789c6532c14d2   worker1<PHONE_NUMBER>8   Claimed      Active   24h
blockdevice-d59dcc24057f4b57fb3fb3fa445503a5   worker3<PHONE_NUMBER>8   Claimed      Active   24h
blockdevice-db4142e757b9b43028bff8fb1b1c77a3   worker2<PHONE_NUMBER>8   Claimed      Active   24h

$ k -n openebs get cspc
NAME              HEALTHYINSTANCES   PROVISIONEDINSTANCES   DESIREDINSTANCES   AGE
cstor-disk-pool   3                  3                      3                  16h

$ k -n openebs get cspi
NAME                   HOSTNAME   FREE     CAPACITY    READONLY   PROVISIONEDREPLICAS   HEALTHYREPLICAS   STATUS   AGE
cstor-disk-pool-729z   worker2    19300M   19300268k   false      1                     1                 ONLINE   16h
cstor-disk-pool-kq22   worker3    19300M   19301090k   false      1                     1                 ONLINE   16h
cstor-disk-pool-xr9s   worker1    19300M   19300264k   false      1                     1                 ONLINE   16h

$ k get sc
NAME               PROVISIONER            RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
cstor-csi-disk     cstor.csi.openebs.io   Delete          Immediate              true                   16h
openebs-device     openebs.io/local       Delete          WaitForFirstConsumer   false                  3d22h
openebs-hostpath   openebs.io/local       Delete          WaitForFirstConsumer   false                  3d22h

$ k get sc cstor-csi-disk -o yaml
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"allowVolumeExpansion":true,"apiVersion":"storage.k8s.io/v1","kind":"StorageClass","metadata":{"annotations":{},"name":"cstor-csi-disk"},"parameters":{"cas-type":"cstor","cstorPoolCluster":"cstor-disk-pool","replicaCount":"3"},"provisioner":"cstor.csi.openebs.io"}
  creationTimestamp: "2022-12-10T16:39:31Z"
  name: cstor-csi-disk
  resourceVersion: "5814198"
  uid: 55e99a8b-2851-4fec-8aa1-5f62c4d7d350
parameters:
  cas-type: cstor
  cstorPoolCluster: cstor-disk-pool
  replicaCount: "3"
provisioner: cstor.csi.openebs.io
reclaimPolicy: Delete
volumeBindingMode: Immediate

The problem is: when I deploy some applications with dis storage class, I get the error:
Events:
  Type     Reason            Age                    From               Message
  ----     ------            ----                   ----               -------
  Warning  FailedScheduling  7m42s                  default-scheduler  0/4 nodes are available: 4 pod has unbound immediate PersistentVolumeClaims. preemption: 0/4 nodes are available: 4 Preemption is not helpful for scheduling.
  Normal   Scheduled         7m40s                  default-scheduler  Successfully assigned nexus/nexus-nexus-repository-manager-5dff6559c-72h5x to worker3
  Warning  FailedMount       110s (x10 over 7m30s)  kubelet            MountVolume.MountDevice failed for volume "pvc-f73ae1f8-66c9-4e4c-b7d7-44e2c80a2a93" : rpc error: code = Internal desc = Waiting for pvc-f73ae1f8-66c9-4e4c-b7d7-44e2c80a2a93's CVC to be bound
  Warning  FailedMount       65s (x3 over 5m37s)    kubelet            Unable to attach or mount volumes: unmounted volumes=[nexus-repository-manager-data], unattached volumes=[nexus-repository-manager-data kube-api-access-2kxnc]: timed out waiting for the condition

Here are the pv and pvc:
$ k -n nexus get pvc
NAME                                  STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS     AGE
nexus-nexus-repository-manager-data   Bound    pvc-f73ae1f8-66c9-4e4c-b7d7-44e2c80a2a93   8Gi        RWO            cstor-csi-disk   9m3s

$ k get pv
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                       STORAGECLASS     REASON   AGE
pvc-f73ae1f8-66c9-4e4c-b7d7-44e2c80a2a93   8Gi        RWO            Delete           Bound    nexus/nexus-nexus-repository-manager-data   cstor-csi-disk            9m10

$ k describe pv pvc-f73ae1f8-66c9-4e4c-b7d7-44e2c80a2a93
Name:            pvc-75ad2ed2-11bd-43cf-baf7-80ad912ff2e2
Labels:          <none>
Annotations:     pv.kubernetes.io/provisioned-by: cstor.csi.openebs.io
Finalizers:      [kubernetes.io/pv-protection]
StorageClass:    cstor-csi-disk
Status:          Bound
Claim:           nexus/nexus-nexus-repository-manager-data
Reclaim Policy:  Delete
Access Modes:    RWO
VolumeMode:      Filesystem
Capacity:        8Gi
Node Affinity:   <none>
Message:
Source:
    Type:              CSI (a Container Storage Interface (CSI) volume source)
    Driver:            cstor.csi.openebs.io
    FSType:            ext4
    VolumeHandle:      pvc-75ad2ed2-11bd-43cf-baf7-80ad912ff2e2
    ReadOnly:          false
    VolumeAttributes:      openebs.io/cas-type=cstor
                           storage.kubernetes.io/csiProvisionerIdentity=1670747336842-8081-cstor.csi.openebs.io
Events:                <none>

OpenEBS version:
$ helm ls -n openebs
NAME   	NAMESPACE	REVISION	UPDATED                                	STATUS  	CHART        	APP VERSION
openebs	openebs  	7       	2022-12-11 08:50:58.164231945 +0000 UTC	deployed	openebs-3.3.1	3.3.0

What is that, and how can I fix it?

#3595


You might need to have better labels because i cant find the right bookmarklet to use. I don't know if its just broken or im using the wrong thing

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to '...'
Click on '....'
Scroll down to '....'
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

their is no reason for a bookmaklet... it runs in the Terminal
?????

Should i be able to put the script into a book mark like some others. Im trying to do some stuff on a school device. all i got is just bookmarklets to have fun. Is are these bookmarklet scripts to begin with?

Thats called a "bookmarklet" (A browser bookmark for chome OS made using java) (I have a couple) and NO its not a bookmark, read how to use the tool on the lower half of the page.


Skill to convert text to Bionic Reading

Describe the contribution to the taxonomy

Bionic Reading it's a modification of a text to increase reading speed. It is basically formatting with bold the initial letters of each words to stand out.
This skill includes 20 examples of this conversion using markdown to format the text so it can do bold.
Bionic Reading page: https://bionic-reading.com/

Input given at the prompt
>>> Convert the following text into Bionic Reading format using markdown: "This is a short text with a single sentence."

Response that was received
This is a [short text](#short-text) with a single [sentence](#sentence).

---

Short text: "This is a short text with a single sentence."

---

Response that is now received instead
Don't have hardware to train the model.
Contribution checklist

[x] Contribution was tested with lab generate
[x] No errors or warnings were produced by lab generate
[x] All commits are signed off (DCO)
[x] The qna.yaml file was linted and prettified (yaml-validator can do both)

Thank you for your contribution to InstructLab! Unfortunately, once a Github repo is made public, all open PRs are automatically closed since they link against a private repo. We have detected that your PR might have been one of the ones affected by this change. If you are still interested in contributing your improvement, please fill out the following short form by no later than May 3rd, and we will get back to you with the additional steps necessary once we have had time to assess the PRs of those still interested:
https://forms.gle/V7SrPPMZDo6iGDYu8


Customs LLMs

Do you have any plans to add configuration options that would allow the use of custom LLMs in future versions?
I recently published a package llm-client that can be very helpful in enabling the support to run other LLM models, including OpenAI, Google, AI21, HuggingfaceHub, Aleph Alpha, Anthropic, Local models with transformers.

👍
It is in the roadmap
https://github.com/Codium-ai/pr-agent#roadmap
Would you like to integrate llm-client into pr-agent?

Why did you mark this as completed? @mrT23

You didnt respond to the question, so i closed the issue.
I can keep it open

I don’t have time to make the integration, but I will be happy to help if needed.

see https://github.com/Codium-ai/pr-agent/pull/172


Make Product name and SKU searchable

If you use the database search driver (ie dont use scout) in the admin the product table is not searchable by SKU or by product name, which are the two ways you are most likely to search this table.
This PR updates that.

@alecritson I think this looks OK, but figured I'd get you to double-check :)


Manually added included URL should not be excluded

It is counter-intuitive that URLs that are specifically added among the included URLs are still ignored if an exclusion rule is found.
Current behavior:

Collect URLs
Add Included URLs
Apply Exclude rules

I would imagine it would be better if the workflow goes as:

Collect URLs
Apply Exclude rules
Add Included URLs

To explain this situation better. If I exclude /gallery/ because I manage that page manually, have a redirect setup there, or whatever other reason, and then I have a random plugin which expects an asset in this following path /wp-content/plugins/random-plugin/assets/mod/gallery/style/gallery-style.css. The asset will be ignored even if added to the explicitly included URL list.

+1

This should not be a blocker anymore. If using WP2Static 7.1.6 and optionally, https://github.com/leonstafford/wp2static-addon-advanced-crawling, if you still find this issue happening, you can either call wp2static_modify_initial_crawl_list filter or directly add/remove URLs by manipulating the CrawlQueue via its static methods.


CU-8693wtx4b: Add code and notebook to compare two models

This is a WIP.
What's there should work, but I'd probably need to add stuff to it still.
Putting it up here in case someone wants to take a look while I'm away on Friday and Monday.
The idea is to be able to compare two different models to see which one should be used.
Hopefully this makes a little bit of sense.
The relevant notebook is here:
https://github.com/CogStack/working_with_cogstack/blob/CU-8693wtx4b-model-compare/medcat/compare_models/model_comparison.ipynb

lgtm


help

Does this occur consistently?
Repro steps:
1.
2.
Action: azureLogicAppsStandard.validateAndInstallBinaries
Error type: Error
Error Message: Unable to write into user settings because the file has unsaved changes. Please save the user settings file first and then try again.
Version: 4.107.4
OS: win32
OS Release: 10.0.26120
Product: Visual Studio Code
Product Version: 1.95.3
Language: en
Call Stack¨NBSP;

Hi @B8nk1028 thanks for raising this issue. I took a look at the error message, and it seems that it doesn't come from our extension, rather it comes from the vscode itself about not saving the changes while updating the settings. When we install the binaries we do update the settings.
I have never seen this issue before. By any chance did you have the settings window from vscode opened while the binaries were installed? Did you updated manually any settings?

so i believe azure broke... but now git is have a problem
i believe i was hit with a root kit. so now ive bought win 11 usb. and am having unlucky success at finding another computer to download a new mbr and drivers
so i will update after i reinstall everything
but my git is now saying rebase error
i got a lot going on here thats all bad it seems feel free to close this. thanks for your help tho


fix(ci): only run the full sync test on mergify queue PRs

Motivation
We want to run the full sync test exactly once before merging each PR.
GitHub API Reference
https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#running-your-workflow-based-on-the-head-or-base-branch-of-a-pull-request-1
Solution

only run the full sync test on mergify/merge-queue/ branches

Review
@gustavovalverde or @dconnolly can review this PR.
Reviewer Checklist

[ ] The full sync test doesn't run in this PR
[ ] The full sync test runs in the Mergify queue PR for this branch

Follow Up Work

make the full sync test faster
require the full sync test before merging Mergify queue PRs

Shouldn't we consider adding Test full Mainnet sync job to Mergify's configuration file as a required check?

Once upon a time we said we wanted the full sync test to always run, but not block merge, because it takes a long time. Is that still what we want?

Our priority is getting CI stable and working, so I'd like to focus on this specific bug fix in this PR.
These questions are about functionality changes. Can you open discussions or tickets for these questions?
Once we have fixed this bug, and the test is stable, we'll have better information to make these decisions. Then we can schedule those changes in a sprint. (I'd like to check with the team before we make changes with a big impact on CI performance or failure rates.)

Shouldn't we consider adding Test full Mainnet sync job to Mergify's configuration file as a required check?

We can't make this change right now, because the job is currently failing. It took more than 6 hours on the most recent PR.

Once upon a time we said we wanted the full sync test to always run, but not block merge, because it takes a long time. Is that still what we want?

That is the current configuration, and it will continue to be the configuration after this PR merges.
I think our decision will depend on how fast and reliable the full sync test is.

The full sync test runs in the Mergify queue PR for this branch

https://github.com/ZcashFoundation/zebra/runs/5458521857?check_suite_focus=true


How to mask "warnings" in compilation

hi，I'm having a problem compiling ceedling. There are a lot of -warnings,

Can you disable "warning" by configuring -project-yaml?

Warnings do not prevent from compiling. Please use preformated code blocks when posting the  output, so that it renders in a bit more readable way.

@Letme
I'm sorry I didn't catch your meaning.
I have a few questions for you.

When you say preformated code blocks, do you mean processing in source code?,  Or in the project-yaml configuration?
Can you demo the pseudo code simply?

Compiler warnings do not prevent anything from compiling. This is just a way that compiler is telling you:

I am assuming something, that is correct in most cases, but it might not be correct in your case - please check if my assumption is correct.

It also means even with 1000 warnings (sadly, I have seen such case), you will get a binary (object) file on the end. Warnings do not abort the compilation.
Preformatted code blocks - sharing log information
Markdown (https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet#code) has a decent option to share code and logs so that they are scrollable. Not just some cut-out screenshot where you can't see anything, because half of the words are removed.
Ceedling detecting memory leaks

Ceedling doesn't do anything with detecting memory leaks. Ceedling is just an aggregator of the tools -> think of it as CMake. Ceedling will assume that if binary is executed and it does not report any passed/failed tests, that it crashed and that there is a memory leak. The message it prints out is exactly like that. There is no way to figure out what happened unless you take gdb, run the already built binary and catch the segfault yourself.
Compilation error and C-Exception have 0 in common. C-Exception is a tool to catch/inject C exceptions within code blocks, the code still needs to compile and be run for this tool to inject various faults into the code and run them inside the tests. It is a helper tool inside the testing framework that allows you to simulate various exceptions -> Compiler error is not an exception at all.
I think you should read: https://github.com/ThrowTheSwitch/Ceedling/tree/master/plugins/gcov#utilities or even https://github.com/ThrowTheSwitch/Ceedling/tree/master/plugins/gcov#gcovr-html-reports


Improving Connect Bones Feature

I added to the connect bones feature by making the code automatically make end bones point in the direction of the parent bone. I also had the code enable connected on the bones if their parent bone only has one child.

It wouldn't affect the armature negatively if any 0 weight end bones are re-oriented as I did in the code. Though if someone does have a foot bone without a toe bone on the end, it would cause the foot bone to point straight down. That is the only problem I can think of. Maybe the code will need a way to figure out if it is a foot bone?

That's why I suggested to only handle the bone as an end bone, if it has no weight on it. This way you can make sure that you have no negative effect on the armature

I will make that update to the pull request. Maybe the fix selected bone ending should be a seperate tool? I will try adding that as a separate button


Switch to JavaScript (ES6) from CoffeeScript

Just out of curiosity: why would you like to move away from CoffeeScript?

I'm doing this now before we write more CoffeeScript and accrue more technical debt.

Upgrade the Beautifiers to use static member properties: https://babeljs.io/repl/#?evaluate=true&lineWrap=false&presets=es2015%2Cstage-1&code=class MyClass {
  myProp %3D 42%3B
  static myStaticProp %3D 21%3B

  constructor() {
    console.log(this.myProp)%3B %2F%2F Prints '42'
    console.log(this.constructor.myStaticProp)%3B
    console.log(MyClass.myStaticProp)%3B %2F%2F Prints '21'
  }
}

let c1 %3D new MyClass()%3B
%2F%2Fc1.initialize()%3B

class MyClass2 extends MyClass {
  myProp %3D 24%3B
  static myStaticProp %3D {222%3A23}%3B

}

let c2 %3D new MyClass2()%3B
%2F%2Fc2.initialize()%3B

class MyClass {
  myProp = 42;
  static myStaticProp = 21;

  constructor() {
    console.log(this.myProp); // Prints '42'
    console.log(this.constructor.myStaticProp);
    console.log(MyClass.myStaticProp); // Prints '21'
  }
}

let c1 = new MyClass();
//c1.initialize();

class MyClass2 extends MyClass {
  myProp = 24;
  static myStaticProp = {222:23};

}

let c2 = new MyClass2();
//c2.initialize();

I started converting the existing code base from CoffeeScript into ES6, however given the features I want to support for Atom-Beautify v1, it may end up simply being a complete rewrite.

Atom-Beautify will be completely rewritten in TypeScript. Work has started: https://github.com/Unibeautify/unibeautify


Space commands not working

The space function is not working with;
--create
--destroy
--focus
They are working correctly with;
--balance
--rotate
The following is the snippits from my skhdrc are not working;
## create desktop, move window and follow focus - uses jq for parsing json (brew install jq)
#shift + ctrl - n : yabai -m space --create && \
#                   index="$(yabai -m query --spaces --display | jq 'map(select(."is-native-fullscreen" == false))[-1].index')" && \
#                   yabai -m window --space "${index}" && \
#                   yabai -m space --focus "${index}"
## Create space on the active display
#alt + ctrl - p : yabai -m space --create
## Delete focused space and focus first space on display
#alt - p : yabai -m space --destroy
## cycle space
##     \
#alt - 0x2A : yabai -m space --focus recent
##     [
#alt - 0x21 : yabai -m space --focus prev
##     ]
#alt - 0x1E : yabai -m space --focus next
Running any of the following commands;
yabai -m space --create
yabai -m space --destroy
yabai -m space --focus recent
yabai -m space --focus prev
yabai -m space --focus next
returns this;
cannot create/destroy/focus space due to an error with the scripting-addition.
I am running version 5.0.2 installed via the command;
brew install koekeishiya/formulae/yabai --HEAD
I'm certain I installed the cirtificate correctly as the instructions didn't seem confusing or have any weird messages.
Please correct me if I'm using any terminology incorrectly, as I'm basicly all self taught through tutorials, documentation, and trial and error.

Same issue here. Focusing simply won't work.

Same problem exists for my m1pro.
macOS 12.6 (21G115)
Darwin 21.6.0

This error also happened me. I disabled SIP, set self-signed certificate, installed yabai HEAD via brew, configure scripting edition with updated hash. but still space command doesn't work

This error also happened me. I disabled SIP, set self-signed certificate, installed yabai HEAD via brew, configure scripting edition with updated hash. but still space command doesn't work

This has been solved. There was just a little mistype for my case..


Closes: #1

Updated validations.py python script.
Fixed the behavior of validate_user function in validations.py for "." or "_"
Fixes #<issue_number_goes_here>

It's a good idea to open an issue first for discussion.

[ ] Tests pass
[ ] Appropriate changes to README are included in PR

@googlebot I signed it!


Using Radzen or MudBlazor for some UI elements

When we need to make multiselection dropdowns and do that statistics page for the Publishers, I suggest that we're using either Radzen or MudBlazor for those elements. For simplicity and also to give the users a good experience, rather than maybe ending up with a wonky chart that we've made ourselves.
I can also try to reaserch for certain NuGet packages only doing those specific things, as we'd have to add the entirety of MudBlazor components and Radzen components, despite us maybe only using few elements.
So I'd like you guys' feedback on this as soon as possible, so that I can look into other solutions if you're against adding huge UI third-party libraries and would rather specific NuGet for the specific purpose. 😊

What's the difference between the two?
Why select one over the other?

They're doing basically the same, they're two big Blazor UI NuGet packages with a lot of UI components there makes handling HTML/CSS/JS much easier through already finished libraries.
They do the same so, I wouldn't say to choose one over the other is better... I think it's a better question which style you like the most of the components that they offer tbh.
If we need a Rich text editor, however, only Radzen offers that but, I don't see any usage of that in this prototype when we don't implement the editing of the front page.

Might wont to give publishers rich text capabilities when writing descriptions for their books ? 🤔
Can you private demonstration on the two styles, so we can look at it and make a choice based on that?

I can link their two documentations or show those when we have the meeting today - I think it'd be a waste of time to set up different demonstrations on the two; So I will show on their documentation examples.
Also, Publishers won't give a description to the books currently - so I don't know if we want to add that and maybe call it summary instead of description? And even then, I wouldn't say that there's needed bold, italic, underline or lists. However, we can talk about that as well. 😊

We've agreed to use one of the bigger libraries for the elements that needs a little more than some simple bootstrap to make the functionality work more smoothly.


⚠ Bofrost.be has degraded performance

In 5ea9790, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1051 ms

Resolved: Bofrost.be performance has improved in cafdc89.


chore(master): release 0.33.6

:robot: I have created a release beep boop
0.33.6 (2022-11-28)
Bug Fixes

use key instead of code for keyboard events (5e919f4), closes #298

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/easyops-cn/docusaurus-search-local/releases/tag/v0.33.6 :sunflower:


Extract logging and offer to clients

Hey! Thanks for this library, @zenazn. One question: it would be great to be able to log using goji's built-in logger so we can track output from each request. What do you think about this? Right now, any other logs don't get the benefit of the request ID and such. It'd be nice to use some global logger like github.com/zenazn/goji/web.Logger or something.
web.Log(c, message)
web.Logf(c, format, args...)

Actually at work I wrote a package that exposes more or less this exact interface :). I'll look into open-sourcing it, but in the meantime feel free to drop me a personal email and I can share a preview of it with you

what I've been doing in my projects is copying the simple logger middleware from goji and putting into my own project and swapping the actually logger with my own (in my case, Logrus)

Any movement here?

Any movement here?

Sorry, haven't gotten the chance to do this yet :/

@parkr : You can use https://github.com/Innovatube/log4go . I have implemented to support goji middleware logging.


What generic/common functions could be added?

Ideas for little functions welcome. Currently I'm just looking at tiny reusable functions that I already wrote in other repositories.

Extracted from https://github.com/pixiebrix/pixiebrix-extension/pull/2317
function printf(string: string, arguments_: string[]): string {
  // eslint-disable-next-line unicorn/no-array-reduce -- Short and already described by "printf"
  return arguments_.reduce(
    (message, part) => message.replace("%s", part),
    string
  );
}
/** Wrapper around the dev tools’ `eval` function to throw proper errors */
export async function devToolsEval(code: string) {
  const [response, error] = await browser.devtools.inspectedWindow.eval(code);

  // Handle Dev Tools API error response
  // https://developer.chrome.com/docs/extensions/reference/devtools_inspectedWindow/#method-eval
  // https://github.com/pixiebrix/pixiebrix-extension/pull/999#discussion_r684370643
  if (!response && error?.isError) {
    throw new Error(printf(error.description, error.details));
  }

  return response;
}

[ ] getTabsByUrl doesn't belong to this module:

https://github.com/fregante/webext-content-scripts/blob/707825400b2a2984901301c5e053a9265282e883/index.ts#L196-L207


Missing/minimal support for table fields that are array types.

It seems that there is currently no support for array types, at least not as far as I can find.
The type returned for array is just the binary data returned from libpg, and not an [Any?]? as I would have expected.
I have managed to figure out how to decode the result, and have added rudimentary support for 1-dimmensional text arrays (OID 1009).
Let me know if there is interest in a pull request, but be warned that I would likely need some guidance to get my hack to an acceptable level for merging.

H @TrondKjeldas, currently, of the three main database plugins supported by SwiftKuery, only PostgreSQL supports the array type and therefore it is unlikely we would consider implementing support for it within the kuery framework.
I am also not clear how I would define a column on a table to be of the array type, do you have an example of how your Swift table is defined so it types the column as an array?

Hi,
I'm not actively working on the project where this was an issue at the moment, but for what its worth here is a bit more info.
This is the database scheme I used:
CREATE TABLE hikes (
    hike_id     SERIAL PRIMARY KEY,
    hike_rev    INTEGER NOT NULL,
    name        TEXT NOT NULL,
    length      REAL NOT NULL,
    description TEXT NOT NULL,
    trackref    TEXT NOT NULL,
    imagerefs   TEXT[4]
);

And this was the Swift implementation for the table:
final class HikesTable: Table {

    let tableName = "hikes"

    let id    =    Column("hike_id")
    let rev   =    Column("hike_rev")
    let name  =     Column("name")
    let len   = Column("length")
    let  desc = Column("description")
    let trackref = Column("trackref")
    let imagerefs = Column("imagerefs")
}

extension Hike : FieldMappable {

  public init?(fields: [String: Any]) {

    var id: Int = 2
    if let id32 = fields["hike_id"] as? Int32 {
        id = Int(id32)
    } else {
        return nil
    }

    let revision = Int(fields["hike_rev"] as! Int32)
    let name = fields["name"] as! String
    let length = Double(fields["length"] as! Float)
    let description = fields["description"] as! String
    let trackRef = fields["trackref"] as! String
    var imagesRef: [String]  = []
    if let imgs = fields["imagerefs"] as? [Any?] {
      for i in imgs {
        if let i = i as? String {
          imagesRef.append(i)
        }
      }
    }

    self = Hike(id:id, revision:revision, name:name, length:length,
            description:description, trackRef:trackRef, imagesRef:imagesRef)
  }
}

Given the lack of support for the array column type in MySQL and SQLite I am going to mark this as closed as it is not something we will pursue at this time.


[#SUPPORT] Upgrade stemcells to fix new L1TF bug

What
New speculative bug in Intel chips, L1TF[1]
Fixed in new stemcells [2]
[1] https://www.intel.com/content/www/us/en/architecture-and-technology/l1tf.html
[2] https://www.cloudfoundry.org/blog/usn-3741-2/
How to review
Code review
Who can review
Not me

Related:
https://github.com/alphagov/paas-cf/pull/1488
https://github.com/alphagov/paas-bootstrap/pull/179

Wait, the hash is wrong

Wait, the hash is wrong

The hash is OK now. See https://deployer.hector.dev.cloudpipeline.digital/teams/main/pipelines/create-bosh-concourse/jobs/concourse-deploy/builds/48


updateManyX is not firing subscription

Describe the bug
Using the example  node-graphql-subscriptions
we can create a post object and subscribe to any of its mutations.
The subscription fires if the mutation is using updatePost, but it's not if using updateManyPosts.
To Reproduce
Steps to reproduce the behavior:

Start the docker container defined in the example above and open the playground at http://localhost:4466/

Create a post/draft for testing

mutation {
  createDraft(
    title:"Subscribed title",
    content:"dummy content",
<EMAIL_ADDRESS>  ) {
    id
    title
  }
}

In my case the post will have an id = 'cjokkxcoc00110806rx76ook9', which I will use later for the subscription.
3. Subscribe
subscription {
  post(where: { node: { id: "cjokkxcoc00110806rx76ook9" } }) {
    node {
      id
      title
      content
    }
  }
}

Execute the two types of mutations in the given order:

mutation {
  updatePost(data:{
    content:"Modified content from updatePost"
  },
  where:{
    id:"cjokkxcoc00110806rx76ook9"
  }){
    id
    title
    content
  }
}

and
mutation {
  updateManyPosts(data:{
    content:"Modified content from updateManyPost"
  },
  where:{
    id:"cjokkxcoc00110806rx76ook9"
  }){
    count
  }
}

You will see the following result in the subscription tab:

{
  "data": {
    "post": {
      "node": {
        "id": "cjokkxcoc00110806rx76ook9",
        "title": "Subscribed title",
        "content": "Modified content from updatePost"
      }
    }
  }
}

Expected behavior
The subscription tab must contain two entries if the last mutation has fired an event, but it contain only the first from updatePost.
Screenshots

Versions (please complete the following information):

OS: Microsoft Windows [Version 10.0.17134.407]
prisma CLI: 1.20.7 (windows-x64) node-v10.11.0
Prisma Server: 1.7.9 (got from Playground settings) from docker container based on image: prismagraphql/prisma:1.18.0

Indeed, I overlooked that statement in the docs, thanks! Unfortunately the doc doesn't explain a workaround or a note it will get supported in future versions. Now we should be very careful when using mass mutations together with subscriptions.


add generic host for each HTTP block (vary by host) to add credentials

add generic host for each HTTP block (vary by host) to add credentials
For example, some credentials might be transmitted via Header (JWT)

as a test , https://api.chucknorris.io/

Using Postman

to get the bearer token, send a POST request to api/registration. The body of this request should contain "blockly"
to test authorization, send a GET request to api/restrictedaccess. This request needs to have an Authorization header set to Bearer
sending GET request to api/restrictedaccess without the Authorization token will generate a 401 unauthorized response

done -see JWT  on https://netcoreblockly.herokuapp.com/blockly.html


[Test] Disable inherits_ObjCClasses.swift on watchOS.

It's failing and it's going to take some more time to figure out why.
rdar://problem/50898688

@swift-ci please test


Update aws-java-sdk-core to 1.12.532

About this PR
📦 Updates com.amazonaws:aws-java-sdk-core from 1.12.459 to 1.12.532
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.amazonaws", artifactId = "aws-java-sdk-core" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "com.amazonaws", artifactId = "aws-java-sdk-core" }
}]

labels: library-update, early-semver-patch, semver-spec-patch, commit-count:1

Superseded by #598.


[4.2] SILGen: Guard against unexpected nulls passed into ObjC overrides.

We want to treat arguments to ObjC override and protocol conformance thunks like "call results", since they might be called from ObjC code that doesn't fulfill its nullability promises in practice. Fixes SR-7240 | rdar://problem/38675815.

@swift-ci Please test

@swift-ci Please test macOS

Build failed
Swift Test Linux Platform
Git Sha - 653da361b7b2f00c1dfbe8e3cf2631cf22febc96

@milseman @phausler You aware of this error on the 4.2 branch?
17:31:12
Foundation/NSString.swift:309:25: error: value of type '_StringGuts' has no member '_isContiguousUTF16'

That change was made to corelibs-foundation master after it landed on swift master. Why would one be on the 4.2 branch and not the other?
@parkera what is the branch coordination strategy?

Build failed
Swift Test OS X Platform
Git Sha - 653da361b7b2f00c1dfbe8e3cf2631cf22febc96

@swift-ci please test

Build failed
Swift Test Linux Platform
Git Sha - 653da361b7b2f00c1dfbe8e3cf2631cf22febc96

Build failed
Swift Test OS X Platform
Git Sha - 653da361b7b2f00c1dfbe8e3cf2631cf22febc96

My understanding was that swift-corelibs-foundation would have a 4.2 branch but that it would auto merge from master for now. cc @shahmishal

Yes, we have swift-4.2-branch now and it auto mergers from master for all swift-corelibs-*.
@jckarter We can close this PR now because we re-branched swift-4.2-branch from master.

@shahmishal cherry-picked this into the necessary branches already. Thanks Mishal!


Relationship method on select is not working

I have tried to use the relationship method on the select and it gives en error

    public static function form(Form $form): Form
    {
        return $form
            ->schema([
                TextInputSelectAffix::make('requested_sum')

                    ->numeric()
                    ->required()
                    ->select(fn() => Select::make('currency_id')
                        ->label('Moneda')
                        ->relationship('currency', 'name')
                        ->required()
                    ),

            ]);
    }

Hello,
The error was on a create screen, did not test it on an edit.
The relation is a standard belongsTo relation
class Calculation extends Model
{
//...
    public function currency(): BelongsTo
    {
        return $this->belongsTo(Currency::class, 'currency_id');
    }
//...
}

Also, if it helps, this method of adding the relationshio works
// ...
TextInputSelectAffix::make('requested_sum')
        ->numeric()
        ->required()
        ->select(fn () => Select::make('currency_id')
            ->label('Moneda')
            ->options(Currency::all()->pluck('name', 'id')->toArray())
            ->required()
            ->live()
        ),


Get rid of the ng12Hybrid flag

Before the ng12Hybrid flag, our wait logic was as follows:
if (window.getAngularTestability) {
  // We are in an ng2 app and should use ng2 testability
} else {
  // We are in an ng1 app and should use ng1 testability
}

This causes problems because sometimes, but not consistently, the ngUpgrade API would publish both testability APIs, causing us to use the ng2 testability API, but really we wanted the ng1 testability API (which is what you use in hybrid apps).  So we introduced the ng12Hybrid flag to resolve this ambiguity.  But was it really ambiguous?  Instead we should have just reversed the original logic:
if (window.angular && !(window.angular.version && window.angular.version.major > 1)) {
  // We are in either an ng1 app or an ngUpgrade app and should use ng1 testability
} else {
  // We are in an ng2 app and should use ng2 testability
}

I propose we do the following:

In master, if ng12Hybrid is not set, used the new check (which makes ng12Hybrid needless)
In beta, rebase the above change in and then remove the ng12Hybrid flag entierly

I like this, and it's a good time to do this kind of breaking change. It would also be nice to just have a single waitForAngular function that detects the Angular version and does the right thing.

@mgiambalvo discussed this a bit offline, and this works fine for waitForAngular, but there's one snag during the bootstrap process: If you see the ng2 testability API, but not the ng1 API, it is possible that ng1 simply hasn't loaded yet.  This is extremely unlikely, and would require the app to be loading ng1 asynchronously after platformBrowserDynamic() was called, as opposed to just having a <script> tag at the top of the page with ng1.  So this is the new proposed solution:

Use the new wait logic instead of the ng12Hybrid flag everywhere except browser.get's testForAngular.
Change testForAngular's check() logic to the following:var definitelyNg1 = !!ng12Hybrid;
var definitelyNg2 = false;
function check(n) {
  // Figure out which version of angular we're waiting on
  if(!definitelyNg1 && !definitelyNg2) {
    if (window.angular && !(window.angular.version && window.angular.version.major > 1)) {
      definitelyNg1 = true;
    } else if (window.getAllAngularTestabilities) {
      definitelyNg2 = true;
    }
  }
  // See if our version of angular is ready
  if (definitelyNg1) {
    if (window.angular && window.angular.resumeBootstrap) {
      return callback({ver: 1});
    }
  } else if (definitelyNg2) {
    if (window.getAllAngularTestabilities) {
       return callback({ver: 2});
    }
  }
  window.setTimeout(function() {check(n - 1);}, 1000);
}

So in English: If ng12Hybrid is set, we know we're looking for ng1 testability.  Otherwise, we use the logic from my first comment to determine if we're looking for ng1 or ng2.  Once we know which API we're looking for, we check only for that API.  This is convoluted, but needed because we don't want protractor to conclude we're on an ng2 app simply because resumeBootstrap is taking a little while to show up.  As long as the user has included ng1 in a <script> tag above all their code, this should always correctly differentiate between ng1, ng2, and upgrade apps.
If the ng12Hybrid flag is set, print a warning.  Almost zero users will need this flag, and that small group can deal with a warning.

Because the ng12Hybrid will always be needed for some users, it will never go away, so there is no longer any breaking change, and this can all be done in master

SGTM.


Fix for #74 - String multiline block with variable

Fix for #74 - String multiline block with variable
Masks multiline string blocks, converts them separately
This is apparently 2 commits behind your recent merges. I hope it doesn't mess anything up?

Im still having trouble keeping all these branches straight. My branches look like spaghetti


tf: skip VM suite when VM is disabled

Please provide a description of this PR:

@howardjohn: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

unit-tests_istio
b1d59238722e0c6a48e5d13075df457413c4590a
link
true
/test unit-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


The isFinal flag is confusing

The initial intent of the isFinal flag was allow consumers of this library to know when LCP was finalized (since the Largest Contentful Paint API does not currently expose this information). However, given #75, the heuristics will become significantly less accurate if we're not tracking scrolls.
The isFinal flag was also a bit weird because it only really applied to LCP. CLS is not final until the page has unloaded, and FCP, FID, and TTFB are always final after the first entry is dispatched.
I've also seen examples in the wild where developers were misunderstanding isFinal and waiting until it was true before sending the data to their analytics provider. That is definitely not what isFinal is for, and something needs to be done to avoid that confusion.

I've also seen examples in the wild where developers were misunderstanding isFinal and waiting until it was true before sending the data to their analytics provider.

Actually, i'm using in this way.   In our monitor SDK, we have tried to  listen to unload event then use sendBeacon api to send vitals data, but we find it is not reliable..
So we wait for the LCP final , as the signal to send vitals data, it will help us collect more user's vitals data.

Actually, i'm using in this way. In our monitor SDK, we have tried to listen to unload event then use sendBeacon api to send vitals data, but we find it is not reliable..

It is not reliable, and it is bad for performance. Do not wait for the unload event (see explanations below):

https://web.dev/bfcache/#never-use-the-unload-event
https://developers.google.com/web/updates/2018/07/page-lifecycle-api#the-unload-event
https://github.com/mdn/sprints/issues/3722#issuecomment-699529808

So we wait for the LCP final , seen it as the signal to send vitals data, it will help us collect more user's vitals data.

In v1, you will not be able to do this. My recommendation is to send the data when the callback fires. The callback will fire at the ideal moment for sending the data, for each metric.
If you want to aggregate the metrics and send it in a single request, then you'll have to do that yourself or use a library like web-vitals-reporter (which I recommend, as it handles this use case very well).
If you want to do it yourself, then I'd recommend sending the data in the visibilitychange event, similar to how the web-vitals-reporter library does it.

If you want to do it yourself, then I'd recommend sending the data in the visibilitychange event, similar to how the web-vitals-reporter library does it.

Nice, thanks for your actionable advice, we will dive in to this libaray.