Unable to Run on MiG Setup

System Info
model: huggyllama/llama-65b
tgi version: 0.6.0
hardware:
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.85.12    Driver Version: 525.85.12    CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA A100 80G...  Off  | 00000001:00:00.0 Off |                   On |
| N/A   34C    P0    44W / 300W |                  N/A |     N/A      Default |
|                               |                      |              Enabled |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| MIG devices:                                                                |
+------------------+----------------------+-----------+-----------------------+
| GPU  GI  CI  MIG |         Memory-Usage |        Vol|         Shared        |
|      ID  ID  Dev |           BAR1-Usage | SM     Unc| CE  ENC  DEC  OFA  JPG|
|                  |                      |        ECC|                       |
|==================+======================+===========+=======================|
|  0    1   0   0  |     19MiB / 40192MiB | 42      0 |  3   0    2    0    0 |
|                  |      0MiB / 65535MiB |           |                       |
+------------------+----------------------+-----------+-----------------------+
|  0    2   0   1  |     19MiB / 40192MiB | 42      0 |  3   0    2    0    0 |
|                  |      0MiB / 65535MiB |           |                       |
+------------------+----------------------+-----------+-----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

deployment: Azure AKS on A100 node with 2 nvidia.com/mig-3g.40gb assigned to each pod
I am trying to shard a 65B model across multiple MiGs on a single GPU but it seems that because the number of GPU devices is still 1 a check fails in the text-generation-inference startup routine.
{"timestamp":"2023-05-30T19:59:20.312138Z","level":"ERROR","fields":{"message":"Shard 1 failed to start:\nTraceback (most recent call last):\n\n  File \"/opt/conda/bin/text-generation-server\", line 8, in <module>\n    sys.exit(app())\n\n  File \"/opt/conda/lib/python3.9/site-packages/text_generation_server/cli.py\", line 58, in serve\n    server.serve(model_id, revision, sharded, quantize, uds_path)\n\n  File \"/opt/conda/lib/python3.9/site-packages/text_generation_server/server.py\", line 143, in serve\n    asyncio.run(serve_inner(model_id, revision, sharded, quantize))\n\n  File \"/opt/conda/lib/python3.9/asyncio/runners.py\", line 44, in run\n    return loop.run_until_complete(main)\n\n  File \"/opt/conda/lib/python3.9/asyncio/base_events.py\", line 647, in run_until_complete\n    return future.result()\n\n  File \"/opt/conda/lib/python3.9/site-packages/text_generation_server/server.py\", line 112, in serve_inner\n    model = get_model(model_id, revision, sharded, quantize)\n\n  File \"/opt/conda/lib/python3.9/site-packages/text_generation_server/models/__init__.py\", line 135, in get_model\n    return FlashLlamaSharded(model_id, revision, quantize=quantize)\n\n  File \"/opt/conda/lib/python3.9/site-packages/text_generation_server/models/flash_llama.py\", line 151, in __init__\n    self.process_group, self.rank, self.world_size = initialize_torch_distributed()\n\n  File \"/opt/conda/lib/python3.9/site-packages/text_generation_server/utils/dist.py\", line 15, in initialize_torch_distributed\n    assert world_size <= torch.cuda.device_count(), \"Each process is one gpu\"\n\nAssertionError: Each process is one gpu\n\n"},"target":"text_generation_launcher"}

Information

[X] Docker
[ ] The CLI directly

Tasks

[X] An officially supported command
[ ] My own modifications

Reproduction
Run with the following env vars in kubernetes
env:
  - name: "MODEL_ID"
    value: "huggyllama/llama-65b"
  - name: "QUANTIZE"
    value: "true"
  - name: "NUM_SHARD"
    value: "2"
  - name: "PORT"
    value: "8080"

With these pod limits
Limits:
  cpu:                     20
  ephemeral-storage:       200G
  memory:                  200G
  nvidia.com/mig-3g.40gb:  2
Requests:
  cpu:                     20
  ephemeral-storage:       200G
  memory:                  200G
  nvidia.com/mig-3g.40gb:  2

Expected behavior
A shard is placed on each MiG

Why do you expect to achieve by sharding on MIGs?

@OlivierDehaene there are situations where a model may span across machines in a MiG-ed fleet where balancing the shards reduces the overall unused memory space. For instance a 60GB model sharded 3 ways could have 4 instances on 3 A100 GPUs whereas un-sharded and un-MiGed you'd only be able to support 3 instances

"Unused" memory space is required to increase the maximum batch size and therefore increase throughput.
Also, the added networking between the shards will greatly impact latency and your overall throughput will be lower even though you have more replicas.
Sharding is not free at all. Depending on the networking scheme used by NCCL and the model size, you can increase the latency per token by quite a bit.
I recommend this paper if you want to read more on the subject: Efficiently Scaling Transformer Inference.
Feel free to make a PR if you still want MIG devices to be support.

@njhill would nvlink be required here since it is technically on the same device?

OK I'm fine leaving this closed then


Add HTML ID's to status page tabs

Thanks for adding IDs to sections in the status page. Can you add for each of the tabs of the status page?  For example, I want to hide the subscribe tab but it doesn't have an ID so I can't use CSS to set its display to none.

Done! Will be deployed by end of the day today


8.5.1 DBCore and other dependency issues

I keep getting dep issues in 8.5.1 where it looks for DBCore in lib and there is no lib, but the file is deployed under src. Loads of other dependency issues too. Are there any specific steps to install it?

um, level-sublevel has no version 8.5.1. did you mean to post this issue on another repository?

Oops sorry i got confused with subkey
On 21 Jun 2015 12:00, "Dominic Tarr"<EMAIL_ADDRESS>wrote:

um, level-sublevel has no version 8.5.1. did you mean to post this issue
on another repository?
—
Reply to this email directly or view it on GitHub
https://github.com/dominictarr/level-sublevel/issues/93#issuecomment-113885152
.


Ease testing of local stubs by extending "MYPYPATH"

Hi!
I would like to propose a fix for #17.
I'm using a stub file and I noticed that pytest-mypy-plugins did not work well in the following scenarios:

Installing a local library with pip install -e .
Running pytest without prior installation of the local library
Updating the local stub file after installation with pip intall .

I think possible workarounds have been implemented (like importing mypy_path and MYPYPATH), but they don't seem to work out of the box. Any reason not to automatically add the project sources to paths searched by Mypy?

@sobolevn I cloned the two repository locally, applied my patch manually, and run the tests suite. Both succeeded. :+1:

Thanks for doing this!


add monochromatic version of LWFS-SBL

circular basis expansion of monochromatic plane wave and point source
extension of driving_function_mono_wfs_vss to handle plane wave and focused source as virtual secondary sources (similar to time domain implementation)
add driving_function_mono_wfs_pwd to handle plane wave decompositions
add pwd_mono_circexp to convert circular basis expansion into plae wave decomposition

So far we have always used 1i as the imaginary number in the toolbox. In your functions you have used 1j. If you don't mind, you could switch to 1i as well.

If have tested it so far by running:
conf = SFS_config;
conf.plot.normalisation = 'center';
sound_field_mono_localwfs_sbl([-2 2],[-2 2],0,[0 2.5 0],'ps',1000,conf)
sound_field_mono_localwfs_sbl([-2 2],[-2 2],0,[0 1 0],'pw',1000,conf)

which resulted in those figures, that look fine to me:

Would it be worth to add a test_localwfs_sbl() function parallel to test_localwfs_vss()?
In addition, we have test_exp_imp(), but nothing for the monaural case.

I fixed the point source. Could you please test again?

The differences are very subtle, but now the figure is exactly symmetric, which was not the case before.

So far we have always used 1i as the imaginary number in the toolbox. In your functions you have used 1j. If you don't mind, you could switch to 1i as well.

fixed in 7dc3abf

Both testing functions are running, but of course I cannot guarantee that all of the resulting figures look like they should look ;)
For test_localwfs_sbl(0) I get a lot of the following warnings, which doesn't happen for test_localwfs_vss(0):
Warning: SECONDARY_SOURCE_SELECTION: 0 secondary sources were selected.
> In secondary_source_selection (line 176)
  In driving_function_mono_wfs_vss>@(X0,XS)secondary_source_selection(X0,XS(1:3),'pw') (line 98)
  In driving_function_mono_wfs_vss (line 108)
  In driving_function_mono_wfs_pwd (line 78)
  In driving_function_mono_localwfs_sbl (line 96)
  In sound_field_mono_localwfs_sbl (line 92)
  In test_localwfs_sbl (line 159)

Both testing functions are running, but of course I cannot guarantee that all of the resulting figures look like they should look ;)
For test_localwfs_sbl(0) I get a lot of the following warnings, which doesn't happen for test_localwfs_vss(0):

The plane wave decomposition used for LWFS-SBL contains angles uniformly distributed on the unit circle. For some (non-enclosing) secondary source distributions, e.g. linear, the propagation direction of some plane waves leads to an empty x0 after secondary source selection.

Should we then disable that warning in the test function or maybe directly in driving_function_mono_wfs_pwd()?

The results for freq_response_localwfs_sbl([0 0 0],[0 2.5 0],'ps',conf) do look as expected, but the large magnitude at low frequency is caused by numerical instabilities. For the time-domain implementation of the point source driving function, we used crossover between conventional WFS (low frequencies) and LWFS-SBL (high frequencies) to circumvent this instabilities. Shall this also be implemented for the mono version?

I would say yes. It's always a good ideas if both domains are implemented as similar as possible.

What is the purpose of the functions in SFS_analysis? Are they meant to just show a good parametrised example? At the moment, the number of plane waves conf.localwfs_sbl.Npw is estimated based on the
given f. It grows linearly with frequency, so it might very high for some iterations in  freq_response_localwfs_sbl. Should we fix the parameters inside freq_response_localwfs_sbl?

If you think there could be better parameter it would be a good idea to fix this in directly freq_response_localwfs_sbl().
For me the functions in SFS_analysis have two purposes:

They show how you could also calculate impulse and frequency response of your system, without using dummy_irs().
The freq_response_* functions are a good way to compare the imp and mono implementations of the toolbox as they calculate the frequency response using the mono functions and you can do the same with dummy_irs() using the imp functions.

For the driving functions it would be a good idea to add a reference.
For driving_function_imp_localwfs_sbl_ps() we could add your EUSIPCO paper and for driving_function_imp_localwfs_sbl_pw() the one from Nara, correct?
What about the mono-frequent driving functions?

For driving_function_imp_localwfs_sbl_ps() we could add your EUSIPCO paper and for
driving_function_imp_localwfs_sbl_pw() the one from Nara, correct?

No, Nara's paper takes another approach. I will add a reference.

What about the mono-frequent driving functions?

The monochromatic implementation covers mostly all aspects of the time-domain implementation, so it can be regarded as a simple inverse fourier transform of the latter.

Could we do a minor release after merging the PR?

Yes, this shouldn't be a problem. Do you have a few days left as there are a few problems under Octave currently with this PR and I would like to include the Voronoi PR as well.

Do you have a few days left as there are a few problems under Octave currently with this PR and I would like to include the Voronoi PR as well.

I will run the examples at Octave, too. Is there a specific issue with some of the functions? The Voronoi PR will still take a while. We have to cover all the (pathological) cases as done for findconvexcone.

The problems under Octave are related to this Ocatve bug.
The output of linkwitz_riley() has not completely symmetric values, which will raise an could not pair all complex numbers error in cplxpair() which is called by zp2sos(). The asymmetry can look like this:
-4.64393188087332e+03 - 5.65864736943455e+03i
-4.64393188087333e+03 + 5.65864736943455e+03i

where the very last value of the real part of the complex number is not identical.
One workaround would be to add the following at the end of linkwitz_riley()
if isoctave
    % Avoid cplxpair error due to numerical problems in Octave,
    % see http://savannah.gnu.org/bugs/?49996
    z = round(1e9*z)*1e-9;
    p = round(1e9*p)*1e-9;
    k = round(1e9*k)*1e-9;
end

but I was not able to test if this would still give the same result at the end as under Matlab (as I have no Matlab today).

There is an additional octave bug. Try:
zp2sos([1],[-1; 1], 1)
zp2sos([1; -1],[-1; 1], 1)

Your changes seem to have fix the problems.
conf = SFS_config;
freq_response_localwfs_sbl([0 0 0],[0 2.5 0],'ps',conf)

Matlab:

Octave

Regarding the zpsos() problem. I haven't looked into the details how you have fixed it, but do you think this zpsos() call in `bilinear_transform() is affected as well?

I ran the test_localwfs_sbl() under Octave and Matlab and it worked well in both cases. I added another commit to disable the zero secondary source selection warning in the pwd driving functions. If you think that is not a good idea we can revert this.
Otherwise this PR is ready in my opinion and I would clean up the commit history and merge if this is fine with you.

FYI: https://savannah.gnu.org/bugs/?51936

Regarding the zpsos() problem. I haven't looked into the details how you have fixed it, but do you think this zpsos() call in `bilinear_transform() is affected as well?

I think any usage for zp2sos() is affected by this problem, because it seems to occur whenever you have an odd number of zeros.

I have updated the commits, if you agree with them I will merge it into master.
I would like to put also some entries in the NEWS file. Would the following fit?
- add monochromatic implementation of LWFS using spatial bandwidth-limitation
- add monochromatic circular expansion functions for ps and pw
- add function for conversion from circular to plane wave expansion
- add freq_response_* and time_response_* for all LWFS methods

I have updated the commits, if you agree with them I will merge it into master.
I would like to put also some entries in the NEWS file. Would the following fit?
- add monochromatic implementation of LWFS using spatial bandwidth-limitation
- add monochromatic circular expansion functions for ps and pw
- add function for conversion from circular to plane wave expansion
- add freq_response_* and time_response_* for all LWFS methods

Yes


fix(pie-icons-react): DSW-000 fix incorrect "types" location in package.json

Hey, we're trying to use @justeattakeaway/pie-icons-react in one of our Typescript projects. When we try and import one of the icons we receive the following error:

error TS7016: Could not find a declaration file for module '@justeattakeaway/pie-icons-react'. <EMAIL_ADDRESS>implicitly has an 'any' type.

I attempted to debug using tsc --traceResolution. It seems to try and load esm/index.d.ts as referenced in the package.json file, however this does not seem to actually exist in the distributed files.
This PR updates the above to instead point to esm/icons/index.d.ts. There is also a copy in dist/icons/index.d.ts, however the esm version seemed more appropriate given it was already looking for a file there. I don't know too much about ES modules though 😅
I've tested this by modifying package.json locally, and the types seem to work as expected:

Replaced by https://github.com/justeattakeaway/pie/pull/1320


The fit method in XGBRegressor (Sklearn API) doesn't support the xgb_model param

While trying to FIT a previously trained model using the Sklearn API of XGBClassifier i got the following error:
TypeError: fit() got an unexpected keyword argument 'xgb_model'
But on the XGBoost documentation for the XGBClassifier it states:

xgb_model (str) – file name of stored xgb model or ‘Booster’ instance Xgb model to be loaded before training (allows training continuation).
http://xgboost.readthedocs.io/en/latest/python/python_api.html

This is how to reproduce the error:
# File to save the model
xgb_file = 'xgb_trained.xgb'

# Instantiate the model
model = XGBClassifier()

# Fit the model
model.fit(x, y)

# Save the model
model._Booster.save_model(xgb_file)

# Fit the model passing
model.fit(new_x, new_y, xgb_model=xgb_file)

The last line throws the following error:

TypeError: fit() got an unexpected keyword argument 'xgb_model'

This was added recently. Please use the current xgboost code from github.

No pip package to upgrade yet?


Use gulp-order to fix incorrectly ordered scripts and styles in minified bundle.

This is an old edgecase: https://discourse.roots.io/t/ordering-stylesheets-before-gulp-concat/3489/6
Asset-builder gives us the arrays in order. We need to drop these arrays into https://github.com/sirlantis/gulp-order
https://gist.github.com/austinpray/15d4514e93217a66e6d3#file-readme-md
https://github.com/dlmanning/gulp-sass/issues/236

we won't be addressing this :warning:


Bump version of celery-docker

See:

https://github.com/openmicroscopy/infrastructure/pull/257
https://github.com/IDR/idr-metadata/pull/237

General nodding during the meeting.


Transactions Not Appearing in Omni Layer Tab

Sent 0.5 BTC on Omni Core Testnet TXID 40a5a43ddf9b3363cb300a3b949d33bc221c389e61ddb1fd67ce08f627a1401e-000 and received 50 OMNI and 50 Test OMNI.  These appear in the Balances tab but not in the Transaction tab under the Omni Layer sub-tab.
Testing with Mac OSX 10.10.5 with Omni Core version v<IP_ADDRESS>-d62b0a9 (64-bit).

Exodus purchases are currently only processed on a low level. Thanks for pointing it out, this is definitely something to tackle!


Cannot read property 'default_langcode' of undefined

hi,
I use Drupal 9.2.2 and Next.js module. The problem after I install and configure next-drupal package and go to node page, I see the next message:
"Cannot read property 'default_langcode' of undefined"
It is possible that JSON doesn't provide data that next-drupal request. How can I resolve this issue? Thanks

Let's check a few things.

Have you configured a preview route?
Is path aliases configured for the node type?
Have you applied the path for subrequests.module? See https://next-drupal.org/docs/quick-start#apply-patch

hey there, having the same issue, I'm trying to run the project without typescript, only js but I can't find the problem

Can you tell me more about the error message? Which line..etc?

We just tagged a new release with a potential fix. Can you pull the latest next-drupal and try again? Thank you.

thank you very much man I managed to solve the problem with a different config on drupal, not sure why but reinstalling and reconfiguring modules fixed the problem. Anyway I will definitely try out this new release.
best regards

Awesome. I'm going to close this issue. Feel free to reopen or create a new one if you have any question. Thank you.


Animation of fill var not happening after the first tap

When I add tap to a group, the first time I see the nice background animation, but afterwards I don't see anything.  I do get the "tap!" printed to the log, so I know the tap is being registered.  I'd be happy to submit a pull request if you can give me a hint where to look.
    group.onTap { _ in
        print("tap!")
        self.valueBG.fillVar.animate(from: Color.white, to: Color(val: 0x333333), during: 0.1)
    }

Thank you very much for pointing out this flaw. We will work on that in the nearest future and let you know as soon as it's done.

aure, issue is fixed in the master branch, welcome to check it out. Thanks again!

Thanks I'll give it a try!


Can now access the batchid and other attributes

Access the batchid and other attributes from an auth with capture and plain capture
Rejenerated some vcr cassettes

+1


[Bug] Authentication error after changing example password

I was testing deploying the minimal galera example with a different password, and hit authentication errors during pod startup.
I edited examples/manifests/config/mariadb-secret.yaml changing the b64 encoded password as follows:
apiVersion: v1
kind: Secret
metadata:
  name: mariadb
data:
  password: aGVsbG93d29ybGQK
  root-password: aGVsbG93d29ybGQK

I deployed the config manifests and the minimal galera manifest:
DOCKER-SHELL1:/tmp/mariadb-operator$ k apply -f examples/manifests/config/
configmap/mariadb created
configmap/mariadb-my-cnf created
secret/mariadb created
secret/user created
DOCKER-SHELL1:/tmp/mariadb-operator$
DOCKER-SHELL1:/tmp/mariadb-operator$ k apply -f examples/manifests/mariadb_v1alpha1_mariadb_galera_minimal.yaml
mariadb.mariadb.mmontes.io/mariadb-galera created
DOCKER-SHELL1:/tmp/mariadb-operator$

The pods fail to start:
DOCKER-SHELL1:/tmp/mariadb-operator$ k get po
NAME                                       READY   STATUS                  RESTARTS      AGE
NAME                                       READY   STATUS             RESTARTS      AGE
mariadb-galera-0                           1/2     CrashLoopBackOff   2 (18s ago)   51s
mariadb-galera-1                           0/2     Init:0/1           0             50s
mariadb-galera-2                           0/2     Init:0/1           0             50s
mariadb-operator-fc74b59cc-sb6qr           1/1     Running            1 (59m ago)   70m
mariadb-operator-webhook-b7c576fdf-wsf7p   1/1     Running            1 (59m ago)   70m
DOCKER-SHELL1:/tmp/mariadb-operator$

With the following error:
Defaulted container "mariadb" out of: mariadb, agent, init (init)
2023-08-02 22:11:14+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:11.0.2+maria~ubu2204 started.
2023-08-02 22:11:14+00:00 [ERROR] [Entrypoint]: mariadbd failed while attempting to check config
        command was: mariadbd --verbose --help
        2023-08-02 22:11:14 0 [Warning] Could not open mysql.plugin table: "Table 'mysql.plugin' doesn't exist". Some options may be missing from the help text
2023-08-02 22:11:14 0 [ERROR] mariadbd: unknown option '--"'

If I deploy without changing the password in the secret, the pods start up without error.
Environment details:

Kubernetes version: v1.25.5
mariadb-operator version: mariadb-operator-0.19.0 v0.0.19
Install method: helm
Install flavour: minimal

You can close this.  It was user error.  The difference between echo password | base64 -w0 and echo -n password | base64 -d

Thanks for reporting anyway @anfechtung 🙏🏻


DNS isn't forwarded to upstream RFC1918 DNS server

Customer has DNS timeouts under the following setup (IPs changed for anonymity)

Upstream DNS set to a <IP_ADDRESS>/24 address in the portal, i.e. <IP_ADDRESS>
<IP_ADDRESS>/24 Resource exists and policy is created
Firezone-managed Relays is selected for the Site (seems to be behind symmetric NAT)
iPhone is on WiFi, local DHCP in <IP_ADDRESS>/24 (non-overlapping range)

Troubleshooting steps taken:

Tested manually resolving names directly against the DNS server, works (dig @<IP_ADDRESS> google.com from iPhone)
Tested using a different upstream with similar packet routing works:

Changed upstream to Cloudflare's <IP_ADDRESS> resolver
Added <IP_ADDRESS> as a resource in the site
Tested again, DNS resolves and uses Cloudflare's resolver

Based on the above, it seems that connlib is having issues forwarding the requests to the upstream <IP_ADDRESS> resolver only.

Needs further testing to ensure the packets aren't going out of the primary interface.

After testing with wireshark on for a bit, I can confirm the behavior if <IP_ADDRESS> is set both as upstream dns server and as a resource is that packets that are outbound from the network extension aren't routed back into it.
This makes sense, since macos doesn't seem to have SO_MARK neither namespaces as a way to have full routing.

Interesting, so this happens because the DNS requests are generated as fresh requests from the Network Extension?
I'm guessing this behavior will be the case for any type of packet generated from the network extension, just DNS.
Is this because they're forwarded using hickory? Would it make a difference if we mangle the destination of the packet to be the upstream DNS server so that the request behaves exactly like the dig @ case

Interesting, so this happens because the DNS requests are generated as fresh requests from the Network Extension?
I'm guessing this behavior will be the case for any type of packet generated from the network extension, just DNS.
Is this because they're forwarded using hickory? Would it make a difference if we mangle the destination of the packet to be the upstream DNS server so that the request behaves exactly like the dig @ case?

I need to do some more testing that I'm still in the midst of, I believe it's because it's indeed a new packet from a UDP socket that needs to be go into the network stack and follow the routing table.
To treat it like the dig @ case we need to handle this edge case in particular by detecting when the upstream server is a resource and writing dns queries to the tunnel instead of using hickory(crafting a new packet or mangling it is the same).

@roop Would you happen to have an idea why these packets aren't routed through the tunnel?

For anyone stumbling across this thread, the issue turned out to be Apple's NECP (Network Extension Control Protocol). One of its primary goals is to prevent VPN packet loops, which means it prevents packets generated from Network Extensions from being routed through the tunnel.
Read more about NECP here.
Since we've verified that DNS requests to upstream Resources work fine when they're initiated by an application instead of the tunnel, the solution we're working on involves mangling the original request to get it through a holepunched channel and connected gateway instead of out the default interface.

As I'm working on the solution for this issue there's some decision we need to make w.r.t multiple upstream servers.
There are two things we need to decide wrt to multiple servers.
One is, if we have multiple upstream dns if we want to function exactly as a stub resolver, we need to be able to keep track of the queries as we detect them and keep a copy of the packet before mangling it and sending, so that after a timeout condition we can query the next nameserver until one respond or the list time outs.
If we do that, also, when we detect a query and we have multiple upstream servers we need to consider that some of them might be resources some of them not, which makes thing a bit more complex.
Having a single upstream DNS would simplify this whole thing greatly but it might also be a huge limitation.

As I'm working there are two things we need to consider wrt to multiple servers.
One is, if we have multiple upstream DNS servers and we want to function exactly as a stub resolver: we need to be able to keep track of the queries as we detect them and keep a copy of the packet before mangling it and sending it out to the gateway. So that after a timeout condition we can query the next nameserver until one respond or we go through all the list of servers.
If we do that, also, when we detect a query and we have multiple upstream servers we need to consider that some of them might be resources some of them not, which makes thing a bit more complex.
Having a single upstream DNS would simplify this whole thing greatly but it might also be a huge limitation.

In the PR related to this issue we will assume a single upstream server and follow up with #3123


WIP: mutation

This adds support for mutation as follows:

RootMapping now has a mutation member of type Mutation (a wrapper for (Query, Env) => F[Result[(Query, Env)]]). All constructors default this value to Mutation.None, which does nothing.
A Mutation performs effects in F and yields a [possibly updated] query and env.
The QueryInterpreter executes the mutation and uses the [possibly updated] query and env to construct the result cursor.
For the doobie and Skunk mappings the updates and subsequent queries run in separate transactions, which I think is actually what we want, given the "mutate and then query" semantics in the spec. In any case not doing it this way would be a significant complication so I'm calling it good for now.

Todo:

[x] Add an database insert test that generates a new key.
[x] Implement database tests for doobie.

It's a pretty small change, which makes me hopeful. More comments inline.

Hm also some test failures. I'll look at those tomorrow.

Ok it's working now. @milessabin I'm ready for your eyes when you have a moment. I'm not super happy with the ergonomics but that seems easy to fix with more Mutation constructors for common cases.

Cool, doobie support is on my checklist above, as well as a few more tests. Will ping you again for a final looksee when those are done.

Alright I added an insert case that elaborates the Select at runtime and moved the test suite so it's shared with both database back-ends. Any last words?

This LGTM! :shipit:


Question about ccache under conda on windows

[ This is not a bug report or a feature request but rather just a question that I was led to, when reading the documentation. ]
I am using nuitka, on Win64, in the following way. The goal is a standalone executable file, created from a .py file such as this example hello.py (which imports and uses only numpy):
def talk(message):
    return "Talk: " + message
def main():
    print( talk("Hello World"))
    from numpy import ones, shape
    print('size = %s' %str(shape(ones([20,3]))))
    input('hit enter when done')
main()

First, I use conda to create a py3.7 virtual environment, in which I include:

nuitka (<IP_ADDRESS>)
m2w64-gcc
numpy, pandas, scipy, matplotlib and other common packages that I tend to use.

Then I get the files get-hints.py, nuitka-hints.py, and hinted-mods.py (all from the github repo) and put them in a folder together with hello.py.
Next, I cd in to the folder, activate my conda env, and from within it I execute

python get-hints.py hello.py

which creates the json file, and

python nuitka-hints.py hello.py

which creates the executable file hello.exe, inside the folder hello.dist (it also creates folder hello.build).
The executable is created and it can be run successfully. This is nice.
However the compilation can be relatively slow (10s of minutes), and the resulting folders are pretty large (100s of megs if including numpy and matplotlib etc). I would like to learn what improvements can be made (to the compilation speed and the size of the resulting hello.dist folder).
I noticed in the documentation there is emphasis on improvements from using ccache, and I also noticed there is a warning during the compilation:

Nuitka-Scons:WARNING: You are not using ccache.

So here is my question:
Is there any way for me to use ccache, while still doing my compilation from within the conda env (note: I don't install gcc outside of what has been included by conda in the virtual env)?
If so, how would I do that?
(If you have other more general suggestions, beyond using ccache, for how I can make improvements they are very welcome too of course.)

Update on this. I added ccache to the set of modules included in my conda env. I no longer get the warning that ccache is not being used. However I have re-run nuitka a few times this way and not noticed much or any reduction in its execution time.
I do still see this warning:

scons: warning: No version of Visual Studio compiler found - C/C++ compilers most likely not set correctly

I wonder if this is related to the way ccache seems to be used, but apparently not reducing execution times.

The version of Nuitka on develop would download ccache automatically, and one that works better than the one in Anaconda, I had to switch, because the version in Anaconda was misbehaving, i.e. hanging endlessly, where the official binary from the ccache project does not.
Using the CondaCC is going to be deprecated soon, I have removed the automatic fallback to it on develop already, since using a gcc 5.2 is not useful as opposed to a gcc 10.2, and that Nuitka will download too.
I am hoping to have things more out of the box working that way, and of course, for me to remove the nightmare of people not being able to pick proper binaries themselves, esp. not for MinGW64.
As for the warning, that is because you didn't specify --msvc in which case, the scan for MSVC is not happening. Right now you are using the CondaCC by means of falling back to gcc as a second attempt. As I always have MSVC on my system, I cannot see that. I am however going to drop the warning, it could easily be misunderstand as a request to install it, where obviously that is not deemed necessary.
And for the caching, Nuitka outputs an overview at the end, that indicates how successful it has been, which indicates it. Of course the C compiler is not necessarily the slow thing, sometimes it is, sometimes it is Nuitka. For the later, the good news is that on develop, there have been very many scalability improvements, all geared at speeding up Nuitka. And then there is #887 for which my analysis found that some modules trigger many more global optimization passes than necessary, which for large programs can bit hard. I hope to solve that issue before releasing 0.6.10, but definitely develop will already be somewhat better for you.

Okay that's fine, I will continue to ignore the Virtual Studio warning if I see it.

The version of Nuitka on develop would download ccache automatically, and one that works better than the one in Anaconda, I had to switch, because the version in Anaconda was misbehaving, i.e. hanging endlessly, where the official binary from the ccache project does not.
Using the CondaCC is going to be deprecated soon, I have removed the automatic fallback to it on develop already, since using a gcc 5.2 is not useful as opposed to a gcc 10.2, and that Nuitka will download too.

So, for future versions 0.6.10 onward, I should plan on:

not needing to include ccache in my conda env (because nuitka won't use it anyway)
nuitka will automatically go get the ccache exe file for me and use it.

Correct?

I am not yet taking steps to disable the existing ccache scan, so as to not break people's environments. But I am sure tempted to do so. There is an option --assume-yes-for-downloads that allows Nuitka to do things non-inactively as well, but since it's cached, it's only relevant for CI.
Right now, I want make sure 0.6.10 doesn't have too many problems with the new tools. But surely I am at least going to add checks for gcc to be at least version 8 or higher soon after that release. Refusing to use the existing tool is a bit harsh, but at least for installation instructions of Nuitka, those are going to become easier.
I will decide this based on feedback I get after release.


Question about FAPE loss

In FAPE loss (algorith28 in AlphaFold2 supply, page 34), why do atom coordinates need to rotate and translate based on frames generated by other atoms? I would be very appreciated if someone could answer the question. Thank U :)

@psp3dcg
I think that :

FAPE loss is just a clamped RMSD on atoms positions under many alignments (Rotations+translations) to make the loss invariant to rotations and translations µ

Also, Let's assume that you have the predicted atom coordinates and the GT atom coordinates, which optimal alignment you will  use ? especially at the beginning of the training where the predictions are chaotic ?

In addition, as mentioned in the Supp Material, FAPE has good properties of a pseudometric  with  the following equivalence FAPE(X, Y) = 0 <=> RMSD(X, Y) = 0

I guess also that the clamping is a MinMax Loss Scaler(0,clamp) which  stablize the loss values and therefore the gradients (no exploding or huge variability) => stable and consistent training


Add SyncAPI support

Motivation
Kontent feature SyncAPI will be released/updated soon and it should be available through DeliveryAPI as well.
Proposed solution
Introduce SyncAPI support into DeliverClient.
Additional context
SyncAPI Documentation
Release date
2023-04-12

Released with SDK v. 17.6.0


[Bug]: Image pulling from private registry does not work on ARM nodes

Description
Image pulling from private repositories (gitlab in my case) is not working on ARM nodes.
The first example manifest as expected. It runs successfully on x86 nodes (nodeSelector amd64).
apiVersion: v1
kind: Pod
metadata:
  name: test
spec:
  nodeSelector:
    kubernetes.io/arch: amd64
  imagePullSecrets:
    - name: gitlabkey
  containers:
    - name: web
      image: registry.gitlab.com/henrydays/service:0.0.1

The ARM64 version does not work. It fails with Error: ErrImagePull dial tcp <IP_ADDRESS>:443: i/o timeout error.
apiVersion: v1
kind: Pod
metadata:
  name: test
spec:
  nodeSelector:
    kubernetes.io/arch: arm64
  imagePullSecrets:
    - name: gitlabkey
  containers:
    - name: web
      image: registry.gitlab.com/henrydays/service:0.0.1

And yes, the image I am pulling is multi architecture. It also works in a microk8s cluster deployed on a Hetzner ARM instance.
Kube.tf file
locals {

  hcloud_token = ""
}

module "kube-hetzner" {
  providers = {
    hcloud = hcloud
  }
  hcloud_token = var.hcloud_token != "" ? var.hcloud_token : local.hcloud_token

  source = "kube-hetzner/kube-hetzner/hcloud"

  ssh_public_key = file("~/.ssh/id_ed25519.pub")
  ssh_private_key = file("~/.ssh/id_ed25519")

  network_region = "eu-central"

  control_plane_nodepools = [
    {
      name        = "control-plane-fsn1",
      server_type = "cpx11",
      location    = "fsn1",
      labels      = [],
      taints      = [],
      count       = 1
    },
    {
      name        = "control-plane-nbg1",
      server_type = "cpx11",
      location    = "nbg1",
      labels      = [],
      taints      = [],
      count       = 1
    },
    {
      name        = "control-plane-hel1",
      server_type = "cpx11",
      location    = "hel1",
      labels      = [],
      taints      = [],
      count       = 1
    }
  ]

  agent_nodepools = [
    {
      name        = "agent-x86-small-node-pool-1",
      server_type = "cx51",
      location    = "fsn1",
      labels      = [],
      taints      = [],
      count       = 1
    },
    {
      name        = "agent-arm-small-node-pool-1",
      server_type = "cax41",
      location    = "fsn1",
      labels      = [],
      taints      = [],
      count       = 1
    }
  ]
  load_balancer_type     = "lb11"
  load_balancer_location = "fsn1"
  dns_servers = ["<IP_ADDRESS>","<IP_ADDRESS>"]
  cni_plugin = "calico"
  use_control_plane_lb = true
}

provider "hcloud" {
  token = var.hcloud_token != "" ? var.hcloud_token : local.hcloud_token
}

terraform {
  required_version = ">= 1.3.3"
  required_providers {
    hcloud = {
      source  = "hetznercloud/hcloud"
      version = ">= 1.38.2"
    }
  }
}

output "kubeconfig" {
  value     = module.kube-hetzner.kubeconfig
  sensitive = true
}

variable "hcloud_token" {
  sensitive = true
  default   = ""
}

Screenshots
No response
Platform
Linux

Turns out this is a gitlab issue. I tried using the docker.com registry and it works.

@henrydays Some hetzner IPs are blacklisted by google cloud container registry (maybe what gitlab uses behind the scenes). You have to contact support and tell them to change the IP of the node affected. Or use the k3s_registries to proxy all container request to a private registry. But good you made it work already.
Anyways if it comes back for some other images, search the error in this project and you will see how people fixed it, but the above two recommendations are basically it.


🙊🙈 Run eslint Travis tests against .ts and .tsx files too 🙈🙊

Turns out that without including --ext *.ts,*.tsx files, eslint will only check JS files
This fixes

prettier v2 formatting
updating .eslintrc to .eslintrc.js because .eslintrc is deprecated
using path to set project path in ts configs
ALL THE LINT ERRORS 🕺

@rcebulko I see. The child prettier configs get precedence over the root. I'll just delete that for ya.

@estherkim Strangely when I tried deleting it, it didn't inherit the root as expected, so I added the prop in https://github.com/ampproject/amp-github-apps/pull/775

@rcebulko hmm that's strange. Worked here, PTAL.


Improvements to the seek command

Made the seek command more flexible and condensed it to a single command in playout_controls.
Fixed a small issue if one tries to seek back to negative elapsed times.
Also removed a one second delay after pressing "play", the connection to mpd was not closed.

And another +1 for noting the develop branch :)


test: migrate check_cfg to snapbox

What does this PR try to resolve?
Part of https://github.com/rust-lang/cargo/issues/14039.
Migrate following to snapbox:

tests/testsuite/check_cfg.rs

This commit allow deprecation for most of the tests currently in this file. This is because the tests either use does_not_contain or perform multiple contains assertion for single tool with macro x!.
Here are some potential options to further eliminate the deprecation:

Remove the x! macro and replace the it with str![] for both Windows and non-Windows platforms.
Modify the x! macro to only generate the --check-cfg part for different platforms and manually compose it with format!.

r? @epage

Failed in CI with the following:
---- expected: tests/testsuite/lto.rs:372:13
++++ actual:   stderr
   1    1 | [COMPILING] foo v0.0.0 ([ROOT]/foo)
   2    2 | [RUNNING] `rustc --crate-name a [..]-C lto [..]`
   3    3 | [RUNNING] `rustc --crate-name b [..]-C lto [..]`
   4    4 | [RUNNING] `rustc --crate-name foo [..]-C lto [..]--test [..]`
   5    5 | [RUNNING] `rustc --crate-name foo [..]--crate-type bin [..]-C lto [..]`
   6      - [FINISHED] `release` profile [optimized] target(s) in [ELAPSED]s
   7    6 | [RUNNING] `[ROOT]/foo/target/release/deps/foo-[HASH][EXE]`
   8    7 | [RUNNING] `[ROOT]/foo/target/release/deps/a-[HASH][EXE]`
   9    8 | [RUNNING] `[ROOT]/foo/target/release/deps/b-[HASH][EXE]`
        9 + [FINISHED] `release` profile [optimized] target(s) in 1m 00s

This should be resolved once #14233 is merged.

#14233 is now merged.

Remove the x! macro and replace the it with str![] for both Windows and non-Windows platforms.

Overall, not a fan of x!.

To add though, we can merge this now-ish (maybe remove some of the weirder does_not_contains) and further improve it in follow up PRs, rather than do it all at once.

To add though, we can merge this now-ish (maybe remove some of the weirder does_not_contains) and further improve it in follow up PRs, rather than do it all at once.

I remove all the -Z. However, if we only want to remove the one within the features_doctest function, please let me know!

The x! macro was added very early the development of the feature in Cargo and has always been awkward, it was meant as a simple way to create the cmd argument without relying to much on the actual argument; in part because the syntax was still evolving and changing. It also was a convenient way to handle the Windows vs non-Windows escaping.
It has serve us well, but since then many things have evolved, --check-cfg is now stable, Cargo has a new testing engine...
In other words, feel free to remove the macro.

The x! macro was added very early the development of the feature in Cargo and has always been awkward, it was meant as a simple way to create the cmd argument without relying to much on the actual argument; in part because the syntax was still evolving and changing. It also was a convenient way to handle the Windows vs non-Windows escaping.
It has serve us well, but since then many things have evolved, --check-cfg is now stable, Cargo has a new testing engine...
In other words, feel free to remove the macro.

Thanks for the explanation and the addition from the author's perspective.

I'll merge as-is and we can handle x! in a follow up
@bors r+

:pushpin: Commit 5a09fcb9f4015f01ec78d5a79ae7293ccd53e03d has been approved by epage
It is now in the queue for this repository.

:hourglass: Testing commit 5a09fcb9f4015f01ec78d5a79ae7293ccd53e03d with merge 0e2ef808c80a1332213cff46be252b0e55f1010d...

:sunny: Test successful - checks-actions
Approved by: epage
Pushing 0e2ef808c80a1332213cff46be252b0e55f1010d to master...


Support LpNormalization operator

doc for the ONNX operator:
https://github.com/onnx/onnx/blob/main/docs/Operators.md#LpNormalization

Please provide a method to create a model with this op or some ONNX models containing this op.


Typo

@kurtfurbush : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@kurtfurbush : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@kurtfurbush - Thank you!

@kurtfurbush - Thank you!


no shadowing in win7pro?

every try to shadow an existing session within win7pro 32bit, results in an error.
all tries are done in real Administrator-Mode, so there shouldn't be any access-problem.
taskmgr (context-menu at user-session): remotecontrol of session failed
shadow : remotecontrol failed: error 120 (this function isn't supported by system)
mstsc /shadow:: the windows-version on this server doesn't support shadowing
as I know, remote-app is only supported in win7ultimate, so maybe shadowing, too?
but: why even the taskmgr shows the option?
any trick to activate support for win7pro?

Windows 7 supports shadowing, there's something wrong with your system then.

well, thanks, you're right.
we've now tried some other systems, were it works well (but only as real administrator). Unfortunately it's not clear, why it fails on some systems, either with the above error, or another one: shadowing a console-session works well, but trying to shadow another rdp-session leads to a break of own connection (can be re-established a moment later)
is there any possibility to debug the mechanism or is the only solution the typical solution (re-installation of windows)?

is there any possibility to debug the mechanism or is the only solution the typical solution (re-installation of windows)?

RDP Wrapper doesn't provide any Terminal Services debugging stuff, so try contacting Microsoft support / tech forums for non-typical solutions.

is there any possibility to debug the mechanism or is the only solution the typical solution (re-installation of windows)?

Windows Debugger

Closed as solved. If not - reopen it.


How to transfer the multipage pdfs to images(png)?

Hi, I am confused that how to transfer the pdf datasets (Deepform、KLC) to multi images with the true key-value GT pairs for each transfered png image? Because the datasets download in DUE-benchmark have no page ID information.

Hello，have you ever solved this problem？ I'm also puzzled with this.

Hi, @Ucas-HaoranWei @Coobiw , for multi-page datasets, we've only used the first page as input so far.


fix: check if already deducted a transaction to avoid 5XX response.

Description
When CreateBillingUsage API is called multiple times with same ID, it doesn't check if already debuted and tries to create transaction entry. This results in DB error ERROR: duplicate key value violates unique constraint,  HTTP status code  500 .
Applied a check to see if the ID is present. If no, it's BAU. If yes, it returns an error mentioning credits already applied with HTTP status code 409.
API Testing

Unique ID

Payload:
 "usages": [
    {
      "id": "cbaef2b4-eb91-44c0-bdc4-eed02666ea70",
      "customer_id": "",
      "source": "",
      "description": "",
      "type": "credit",
      "amount": "2",
      "user_id": "",
      "metadata": {},
      "created_at": "",
      "updated_at": ""
    }

Response status:  200 OK

Duplicate ID

Payload:
 "usages": [
    {
      "id": "cbaef2b4-eb91-44c0-bdc4-eed02666ea70",
      "customer_id": "",
      "source": "",
      "description": "",
      "type": "credit",
      "amount": "2",
      "user_id": "",
      "metadata": {},
      "created_at": "",
      "updated_at": ""
    }

Response:
status - 409 Conflict
body -
{
    "code": 6,
    "message": "failed to deduct usage: credits already applied",
    "details": []
}

Pull Request Test Coverage Report for Build<PHONE_NUMBER>4
Details

8 of 11 (72.73%)  changed or added relevant lines in 2 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage increased (+0.006%) to 9.468%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

internal/api/v1beta1/billing_usage.go
0
3
0.0%

Files with Coverage Reduction
New Missed Lines
%

internal/api/v1beta1/billing_usage.go
1
0.0%

Totals

Change from base Build<PHONE_NUMBER>8:
0.006%

Covered Lines:
10369

Relevant Lines:
109515

💛  - Coveralls


Feature Request: React Presentation Layer

Thank you Jason and the community for the brilliant work on this solution. I just finished watching your latest NDC talk and I think it's brilliant.
I was wondering whether anyone is forking this to make a version with React as the front-end? I will try and give it a go but I am really new to it.
H.

I also just recently saw the NDC Sydney 2019 talk. Since it’s mentioned, we should link to it.

Thank you Jason and the community for the brilliant work on this solution. I just finished watching your latest NDC talk and I think it's brilliant.

Thanks so much, really appreciate the feedback. 😄

I was wondering whether anyone is forking this to make a version with React as the front-end? I will try and give it a go but I am really new to it.

This is something that I'd really like to see happen next year. In fact, it would be great to have front ends for Angular, React, Vue (my favourite), and Blazor.
If you want to learn more, take a look here - https://docs.microsoft.com/en-us/aspnet/core/client-side/spa/react?view=aspnetcore-3.1&tabs=visual-studio. Essentially, there are two React templates - React and React with Redux. The React template is most suited, since it supports authentication with ASP.NET Core Identity + IdentityServer out of the box.

I think i just about got React to work with it, but didn't implement the Todo app logic. Currently battling with integrating Azure AD sign in with Identity Server in the template. Failing at it but will grind it until I succeed XD!!
I have so much to learn :(


Unable to find libclang

Describe the bug
When building on a system that lacks libclang, the following error is reported from cargo build.
user@dunx4:~/projects/oxigraph$ cargo build
   Compiling oxrocksdb-sys v0.3.19 (/home/user/projects/oxigraph/oxrocksdb-sys)
   Compiling rustls v0.21.6
   Compiling pyo3-macros v0.19.2
   Compiling sparql-smith v0.1.0-alpha.4 (/home/user/projects/oxigraph/lib/sparql-smith)
   Compiling pyo3 v0.19.2
error: failed to run custom build command for `oxrocksdb-sys v0.3.19 (/home/user/projects/oxigraph/oxrocksdb-sys)`

Caused by:
  process didn't exit successfully: `/home/user/projects/oxigraph/target/debug/build/oxrocksdb-sys-beb14f6200003263/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-changed=api/

  --- stderr
  thread 'main' panicked at 'Unable to find libclang: "couldn't find any valid shared libraries matching: ['libclang.so', 'libclang-*.so', 'libclang.so.*', 'libclang-*.so.*'], set the `LIBCLANG_PATH` environment variable to a path where one of these files can be found (invalid: [])"', /home/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/bindgen-0.66.1/lib.rs:604:31
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...

To Reproduce
Steps to reproduce the behavior:

Which version of Oxigraph are you using? On which platform? - main branch as of the filing of this report, Ubuntu 22.04.3 LTS
A command line or a code snippet that triggers the bug. - cargo build

To fix
For Ubuntu 22.04.3 LTS, installing the libclang-dev package resolves the issue: sudo apt install libclang-dev. This is a simple fix, but it may be worth noting in the documentation for building. In addition, if there is a dependency on a sepecific version of libclang, it might be more robust to specify that.

Thank you for reporting this! Indeed. That would be great to mention in the README that clang is required for building Oxigraph. My guess is that quite old Clang versions should work but I have not carefully checked what is the minimum version.


Develop ASPIRE2 Processor Adapter

Description
Develop a processor base adapter that can be used to simplify the use of the ASPIRE2 supercomputer.
Outcomes

[ ] Implemented and tested ASPIRE2 adapter base class

Obsolete.


add interface ClientProvider for EmulationManager

We can see from the Emulated State that It used ClientProvider interface as its parameter and we actually passed EmulationManager instance. So we better add interface ClientProvider to EmulationManager.

Did you add tests for your changes?
If relevant, did you update the documentation?
Summary

Does this PR introduce a breaking change?

Other information

failed attempt


Multiple Affiliations per author

Thank you for the cool template. I tried it out and realised that while it nicely groups authors with the same affiliation, it does not seem to allow authors to have multiple affiliations. Is there a way to implement this, for example something like:
      (name: "Niklas Luhmann",
      affiliation: ["University of Pennsylvania", "Research Center Somewhere"],
      orcid: "0000-1111-1111-1111")

Thanks for the issue. You're right, multiple affiliations are currently not supported. Will add this for the next version

@LeSasse, I added this feature with the latest commit (19df400). You can either use the latest version directly here from github or wait until my pull request (typst/packages#1202) was approved and the update propagated to the typst universe.
Note that you have to update your import statements as well:
#import "@preview/apa7-ish:0.2.0": *

Thanks for the quick implementation! Will clone it from here as I write locally.


fix(api-usage-plan): update cfn templates with api usage plan

Issue #, if available: 396
Description of changes:
Update Cfn templates to match the expected results for CDK v2 and V1
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

AWS CodeBuild CI Report

CodeBuild project: codebuildgithubautobuildPro-fkVQbXRiQi6A
Commit ID: 1babdaa464b057689a8da0e3c066a739473af359
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: codebuildgithubautobuildPro-fkVQbXRiQi6A
Commit ID: 4b0a0c62141fe2c4324a85015ca76132856750c2
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


Keyboard remapper icon - MDL2 E765

Per icon team, use E765

@crutkas I think F210 or F73D would be better. The gear indicates settings. What do you think?


Unexpected behavior of cron filters

I'd like to run certain pipeline on cron trigger only. Here's my pipelines configuration:
[
    {
        kind: "pipeline",
        name: "first",
        steps: [
            { name: "uname", image: "alpine", command: ["uname","-a"] }
        ],
        trigger: {
            event: ["push"],
        },
    },
    {
        kind: "pipeline",
        name: "second",
        steps: [
            { name: "env", image: "alpine", command: ["env"] }
        ],
        trigger: {
            event: ["promote"], // otherwise cron doesn't appear in output - see #2730
            cron: ["sample"]
        },
    }
]

After push initiated build I'm adding cron job with
drone cron add org/repo sample "0 */15 * * *"
Expected behavior:  pipeline second is executed every 15 minutes.
Observer behaviour: pipeline first is executed every 15 minutes. No signs of second.
Also drone build promote org/repo BUILD_NUMBER somewhere doesn't start second pipeline either. Logs:
... trigger: skipping pipeline, does not match event ...
... trigger: skipping pipeline, does not match cron job ...

:wave: hey there, per our issue template, for end-user support please use one of our support channels, such as our chatroom or our mailing list.


Revert colors back to 0.6.2

It was causing colors to not show in grunt.
@shama or @vladikoff please could you review and merge.

I doint know why this is failing since I'm only going back to a version that was working before.

Tracking https://github.com/Marak/colors.js/pull/154 for a fix.

It's unlikely it will be merged since there hasent been a commit since September 2015.
I've submitted the pull here https://github.com/gruntjs/grunt-legacy-log/pull/23 that it uses that commit

Appveyor issues:
Running "nodeunit:util" (nodeunit) task
Testing index.js.............F.................
>> Log instance - writeflags
>> Error: 'Flags: a, b\n' == 'Flags: \u001b[36ma\u001b[39m, \u001b[36mb\u001b[39m\n'
>> at stdoutEqual (test\index.js:26:8)
>> at Object.exports.Log instance.writeflags (test\index.js:210:5)
>> at Object.exports.Log instance.setUp (test\index.js:57:5)

>> Log instance - writeflags
>> Error: 'Prefix: a, b\n' == 'Prefix: \u001b[36ma\u001b[39m, \u001b[36mb\u001b[39m\n'
>> at stdoutEqual (test\index.js:26:8)
>> at Object.exports.Log instance.writeflags (test\index.js:213:5)
>> at Object.exports.Log instance.setUp (test\index.js:57:5)

>> Log instance - writeflags
>> Error: 'Prefix: a, b=false, c=0, d=null\n' == 'Prefix: \u001b[36ma\u001b[39m, \u001b[36mb=false\u001b[39m, \u001b[36mc=0\u001b[39m, \u001b[36md=null\u001b[39m\n'
>> at stdoutEqual (test\index.js:26:8)
>> at Object.exports.Log instance.writeflags (test\index.js:216:5)
>> at Object.exports.Log instance.setUp (test\index.js:57:5)

Warning: 3/209 assertions failed (125ms) Use --force to continue.

@vladikoff yep that's because we updated to colors 1.1.2 which broke some programes such as ones based on Linux like git for windows and cygwind. Plus powershell I doint think works with colors 1.1.2 because of the new check that was added that broke it meaning it wasn't really testing colors because it would switch colors off because of the check. But https://github.com/Marak/colors.js/pull/154 should fix it but is unlikely to be merged any time soon looking at the commits the last one was in septemeber 2015.
But appveyor was added after we updated colors to 1.1.2 so if we added it when we were using 0.6.2 it would likely have been broken.


Trained models have a metallic sound..?

Hello, I've recently reinstalled Mangio RVC and now when training new models the output is always a metallic/robot voice. Is there any solution?
https://github.com/Mangio621/Mangio-RVC-Fork/assets/82603798/70adbb29-a3d4-442f-aa1e-e9fd74289510

Did you resolve it, I have the same exact issue?


Too many levels of nesting

Hello!
I'm trying to run the static Java method Analyzer.analyzeStatement over a quite long select statement with multiple CTEs, and I'm getting the following error:
Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message had too many levels of nesting.  May be malicious.  Use CodedInputStream.setRecursionLimit() to increase the depth limit.
The catalog is properly built and everything works perfectly with smaller queries.
Is it possible to increase this recursion limit or a workaround to parse long SQL statements?
Thank you and Merry Christmas!

Did you find a workaround?
As I also have this issue.

Unfortunately there's no easy way for this, as the error is from the proto-lite library - so there are two options a) update the ZetaSQL core to increase the recursion level. b) recompile ZetaSQL with a custom version of proto-lite.


accepting "symbol" vs "stock" in incoming POSTs

See https://starfighter.readme.io/docs/place-new-order
Officially, the correct JSON fieldname when POSTing to the server is "stock" not "symbol". The official server accepts both though. But when I fixed this issue in my own client code, I could no longer place orders on Mockfighter: I believe it only accepts "symbol". (My ability to read LISP is non-existant.)
User cite-reader told me about this.

Awesome! The server should accept symbol and stock now.


[Feat] - Added Ignored Paths to Deploy CI

Description:

This PR updates the deploy CI to ignore changes to the below-mentioned files

'README.md'
'LICENSE'
'CODE_OF_CONDUCT.md'
'.vscode/**'
'.husky/**'
'.env'
'.gitignore'

If files apart from the above ones change, then the CI will be triggered.

This PR also adds a workflow_dispatch trigger in case the deploy fails and the maintainers/admins want to trigger it manually

Hello, I am a code review bot on flows.network.
It could take a few minutes for me to analyze this PR. Relax, grab a cup of coffee and check back later. Thanks!

LGTM


Storagecluster: generate mgr spec in a function

This also updates the count of MGRs that are created for an arbiter
deployment to 2.
Signed-off-by: Raghavendra Talur<EMAIL_ADDRESS>

@obnoxxx @jarrpa I have updated the PR now. It was in WIP because I had not added the resources limits for the mgr sidecar container. It is added now.

Looks like you may have pushed the wrong commit. You're missing some curly braces and didn't clean up a merge conflict.

Looks like you may have pushed the wrong commit. You're missing some curly braces and didn't clean up a merge conflict.

Thank you. Updated now.


Add 'endpoint' directive to global api info.

Signed-off-by: Dane Jensen<EMAIL_ADDRESS>

LGTM


Test edit to check CODEOWNERS

try nr 2

Only works if you're not the PR submitter, and if the reviewer has commit rights - so hard to test on a fork.


[Feature/ChangedFiles]#125 Add support for getting changedFiles from PR as resource

Queries the V4 Github API for a list of ChangedFiles in a PR and writes them alongside with version.json and metadata.json as a plain text file called changedFiles into the .git/resource folder

Thanks for the PR, this looks great! Aside from the nitpicks in my comments, I'm wondering how last works for listing PR files in Github V4, and thinking that we'll probably be required to do pagination in order to have a complete list of modified files and not just some of them. If we do pagination however, we'll also have to make this functionality optional via a flag, since it would increase the costs of the operation (ratelimits).
If we make this optional, it might make sense to just move the implementation to ListModifiedFiles() so we can use it for path and ignore_paths in check as well?

I do agree with you, we would need pagination. Right now it is limited to about the 100 last files in the list of changed Files.
However the number of commits then would also need to be paginated, which is not done right now.

However the number of commits then would also need to be paginated, which is not done right now.

Does this mean that the list of modified files is per commit, and not for the PR as a whole? I.e., to get a complete list for the entire PR we would have to paginate all the modified files for every commit?

Sorry for being unclear. The List of changed files is per PR.
I just wanted to point out the big TODO in the code about pagination of commits.

Sorry for being unclear. The List of changed files is per PR.
I just wanted to point out the big TODO in the code about pagination of commits.

Ah I see, thanks for clarifying. The TODO about pagination would be nice to fix also, but I think for now we can just focus on having pagination when listing the modified files since it is more likely that users will have 100+ modified files? What do you think about making the functionality optional though, and maybe moving the implementation to ListModifiedFiles()?

I will look into pagination. However I would rather not move it into ListModifiedFiles() as the information is really most useful when the repo gets checked out and the information is written to disk. I think check stay a relatively short operation.

Rebased on current master

Quite a big change, but the fetching of files now supports pagination

@itsdalmo Getting the changed file list from the PR is now optional and supports pagination.

@itsdalmo I think I addressed now all your comments

The travis build timed out out of unknown reasons. Please retrigger @itsdalmo

Fixed all except the append stuff. Good enough? @itsdalmo


Replication

Add a single dataset file for original texts.
Add scripts for replication.
Add configs for each run.

OK it's probably ok to merge this but note a few things.

We should sunset this repo, and make the new repo llmexperts the Python package that runs the llm scripts, with configuration options for prompts and input and output files. This separates our project from the Python engine to produce summaries and scales, thus making it more generally available for other users.

All of our data and prompts need to go into the repo Replication. When adding the replication code, put the code and data into the appropriate locations for that repo, and make each step needed to get to the next step fully reproducible. No steps should be omitted.

Make sure that summaries and scaling are separate steps.

Input files ought to be capable of being in folders, or possibly lists of files from folders. We only have one folder of party manifestos and one of coalition manifestos, so if we partition this into prototype runs we ought to have a way to specify which files we will use.


Add content for fenwick tree algorithms in c++

Select the Day of Content
others
What information is missing?
I would like to add content for fenwick tree algorithms and examples in C++.

@subhadipbhowmik  please assign this task to me.

@subhadipbhowmik Please assign it to me I want to work on this issue. Also, please add the label GSSOC and the level that you find appropriate for this issue. thank you!