[RayService] HTTPProxies don't update when replica crashes and recovers

Search before asking

[X] I searched the issues and found no similar issues.

KubeRay Component
Others
What happened + What you expected to happen
When a replica crashes and recovers in Serve, usually the HTTPProxies get updated, so they can route requests to the newly updated replica. This seems to be Serve's behavior on a local cluster. However, in KubeRay it seems that when replicas crash and recover, the HTTPProxies aren't updated.
However, it does seem that handles are updated, so you can get a deployment handle and successfully query all replicas using the handle after they crash and recover.
Reproduction script
Local Cluster (works)
For example, this application takes in a query and returns the process ID of the replica that processes it. We can create a config and run it on a local Ray cluster to test its behavior:
# File name: config.yaml

import_path: "sleepy_pid:app"
runtime_env:
  working_dir: "https://github.com/ray-project/serve_config_examples/archive/42d10bab77741b40d11304ad66d39a4ec2345247.zip"
deployments:
  - name: SleepyPid
    num_replicas: 6
    ray_actor_options:
      num_cpus: 0

$ ray start --head
$ serve deploy config.yaml
$ curl localhost:8000
7597

We can also kill one of the replicas, and once the replica recovers, it continues processing requests:
$ ray list actors --filter "class_name=ServeReplica:SleepyPid"
...
Table:
------------------------------
    ACTOR_ID                          CLASS_NAME              STATE    NAME                               PID  RAY_NAMESPACE
 0  3d3bcacf85e60fe1cd300a0f01000000  ServeReplica:SleepyPid  DEAD     SERVE_REPLICA::SleepyPid#GeZtqh   7596  serve
 ...

$ python
>>> import ray
>>> replica_handle = ray.get_actor("SERVE_REPLICA::SleepyPid#GeZtqh", namespace="serve")
>>> ray.kill(replica_handle)
>>> exit()

Once the replica recovers, we can continue using curl and see that there are 6 different PIDs, meaning that the proxy continues to route requests to 6 different replicas as expected.
On KubeRay (doesn't work)
We can run the same application on KubeRay, and we can exec into one of the KubeRay pods and kill replicas just like we did locally. We can use requests to query the deployment over HTTP:
$ python

>>> import requests
>>> requests.get("http://localhost:8000").json()
7597

However, when we kill replicas, even after they recover, the HTTPProxy doesn't route requests to the new replicas. Strangely, if you query the deployment via ServeHandle, the requests do get routed to all the replicas. You can get the handle and query it like this:
$ python

>>> import ray
>>> from ray import serve
>>> handle = serve.get_deployment("SleepyPid").get_handle()
>>> ray.get(handle.remote())
7597

Note: I used the same config file as #615 when I discovered this issue. If that config is incorrect (perhaps due to a misconfiguration with GCS FT), then that might be the root cause.
Anything else
I ran into this issue on Ray 2.0 with the KubeRay operator. I used the config from #615 when I ran into this issue. That config includes GCS FT, but it may not be done correctly, so that might be the root cause of the issue.
Are you willing to submit a PR?

[ ] Yes I am willing to submit a PR!

I tried reproducing the issue using the fruit example without fault tolerance, but the HTTPProxy updated correctly. I think the problem is likely in the way I configured GCS FT, so I’ll close the issue for now.


Privacy whitepaper

Small updates to whitepaper, family safety link, sentence for sync, and sentence for feedback.

Docs Build status updates of commit c5028a8:
:x: Validation status: errors
Please follow instructions here which may help to resolve issue.

File
Status
Preview URL
Details

:x:Error

Details

[Error] Cannot sync git repo to specified state. Please make sure the pull request is mergeable. Details: Git merge results in conflict, repository remote url: https://github.com/MicrosoftDocs/edge-developer, working directory: W:\2hzv-s, target branch: HEAD, target branch commit: 38e6b8323a3d8a1dbc7095d1dd622c2b2626e6c1, source commit: c5028a8978fa52ce9dd5b9e21a1b804ebc78a105.

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Docs Build status updates of commit e083117:
:warning: Validation status: warnings
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Closed.
Duplicated in #540.


Corrected usage context in description of PointInCell regrid scheme.

The oneliner comment is wrong -- coords no longer need to be orthogonal.

@SciTools/iris-devs - I'd love to be able to move quicker on these types of PRs. Low risk (doc change), small and well defined. In these cases, I encourage you to simply merge and move forwards. To be completely clear, I also include myself in the list of people who could have moved this forwards quicker (by watching the PRs notifications).
I've started labelling issues/PRs to help with this kind of triage, and encourage everybody to get involved in the labelling effort. It should allow us to see these kinds of PRs and get on to them quicker.
 I'm hopeful we've broken the back of those now with the reduction in number of processes being used for testing (this was due to our improved parallelism through dask).


Compare with previous commit

Hi,
I work on a project with multiple microservices, packaged as docker images. We use the monorepo approach, and are happily using gib for testing on our pull-requests.
However on our master branch we rebuild all the microservices and push them on the docker registry. We would like to be able to run gib by comparing with the previous commit, in order to know which microservice we must build and deploy.
Would you be open that we work on such a feature and propose a pullrequest ? If so, do you already have some guidelines ?
Thank you in advance,
Vincent

Hi @vincent-artea, thanks for getting in touch!
Have you tried this already?

I've updated the README mentioning HEAD~1 in the referenceBranch section.


Last bookmarked on top of the list

Thank you very much for the great plugin.
The new "search bookmarks" function is really fantastic. If I may, I would like to make a small request. Would it be possible to sort the bookmarks differently when called up? For example, I would need an overview that would include something like a "last bookmarked" function. When I then call up your new "search bookmarks" command, the bookmark that I last added to the bookmarks will be displayed at the very top of the list. That would be great.

Thanks for your request!
I have made sure that this is possible.
I'll consider it, please wait for a while.

Thanks for waiting, I released it in v0.5.1.
Try it if you like.


Raspberry

Tested built (v<IP_ADDRESS>) on Raspberry Pi with success.

Thanks for the pull request. However, please don't submit merge-only pull request with main branch, we do it regularly before the release. The pull request will need new features or fix current mistakes etc. Thanks again.


[WFCORE-5236] Upgrade WildFly Elytron to 1.14.1.Final

https://issues.redhat.com/browse/WFCORE-5236
https://github.com/wildfly-security/wildfly-elytron/compare/1.14.0.Final...1.14.1.Final
    Release Notes - WildFly Elytron - Version 1.14.1.Final

        Component Upgrade

[ELY-2050] -         Update bouncycastle to 1.67

        Enhancement

[ELY-2049] -         Add trace capability to o.w.s.m.WildFlySecurityManager findAccessDenial

        Bug

[ELY-2057] -         No acceptedIssuers is sent when CRLs are configured

[ELY-2058] -         Accepted Issuers should not be manually configured in X509RevocationTrustManager

[ELY-2059] -         SSL Tests Failing on Zulu JDK 11

[ELY-2063] -         MaskedPassword interface contains 8 algorighms those are unavailable on IBM JDK8

        Task

[ELY-2056] -         Add a GitHub Action Workflow for CI

[ELY-2060] -         Move CA set up from SSLAuthenticationTest into re-usable util

[ELY-2061] -         The SASL BaseTestCase is no longer a general base

        Release

[ELY-2066] -         Release WildFly Elytron 1.14.1.Final

Core - Full Integration Build 10233 outcome was FAILURE using a merge of d18146aca3dbd1d5533c39565f384c4eb5ee0fe5
Summary: Tests passed: 6778, ignored: 109; exit code 137 (Step: Build & test full (Maven)) (new) Build time: 02:41:15

Core - Full Integration Build 10233 outcome was FAILURE using a merge of d18146aca3dbd1d5533c39565f384c4eb5ee0fe5
Summary: Tests passed: 6778, ignored: 109; exit code 137 (Step: Build & test full (Maven)) (new) Build time: 02:41:15

@darranl the KeystoreRealmTestCase failures on Windows looks genuine. Could you please have a look?

@darranl the KeystoreRealmTestCase failures on Windows looks genuine. Could you please have a look?

@darranl thanks

@darranl thanks


error-regression

Thanks, the code seems great!
One doubt about the cli-design now in retrospective: Is it better to implement this workflow as a new command (which is what I've asked you to do and what you've done)? Or would it be better to have it as an argument to the behavior (currently called regression) command?
Quick pros for new command:

Leaves more flexibility to customize each command differently via different args if we want to. Can you think of different customization we might want to have?
More flexibility also in general.

Quick pros for argument to behavior (currently called regression) command:

Would reflect clearly that this is the exact same thing with the difference of reducing the Merlin response to success/error.
Would allow getting both outputs (full Merlin response and this new success/error response) in one single merl-an run, analyzing each sample only once. So if we want that workflow, that would be better for the performance of the CI.
Less code.

This has been only the first things that have come to my mind. What's you opinion in this?


PG - not retaining reference particle?

Hello,
I've recently updated Turing from v0.13.0 to v0.15.18 and noticed that PG() does not work the same way as before, specifically when it comes to retaining reference particles (I'm using Julia 1.4.2 for both). Using an example from Turing Guide: https://turing.ml/dev/docs/using-turing/guide
@model function gdemo(x, y)
    s ~ InverseGamma(2, 3)
    m ~ Normal(0, sqrt(s))
    x ~ Normal(m, sqrt(s))
    y ~ Normal(m, sqrt(s))
end

# Use only one particle for PG
c2 = sample(gdemo(1.5, 2), PG(1), 1000)

I tested it out with only one particle, where PG should retain its original particle throughout the iterations. This is the case for v0.13.0, but not for v0.15.18 (plot of m shown below). I'm wondering if this is due to resetting VarInfo before creating the reference particle (line 282 onward in Turing.jl/src/inference/AdvancedSMC.jl). Please let me know if this is intended or I'm mistaken somewhere, as I'm just a Turing/Julia beginner :')
Old Turing ver:

New Turing ver:

Thanks for opening the issue and providing this clear and simple example! Recently, we started to extract the particle sampling logic to a separate package AdvancedPS.jl such that it can be used even without Turing. Currently, this is not possible (in a nice way) and both packages still not clearly separated. I assume that during the refactoring we (or probably rather I :smile:) broke something accidentally.
In principle, https://github.com/TuringLang/Turing.jl/blob/46c05895d7e51028211a856c78b1cca60838c5d4/src/inference/AdvancedSMC.jl#L294 should ensure that the reference particle is retained but it seems this is not the case anymore. Maybe because the VarInfo object is reset before. I'll have a closer look at the example and check why it fails.

I just checked the implementation in Turing 0.13, and the problem seems to be that the VarInfo object is reset before the reference particle is created. In 0.13 it was done in the correct order: https://github.com/TuringLang/Turing.jl/blob/98869cf5f4713fde8d1473576ec1afb72314efd9/src/inference/AdvancedSMC.jl#L238-L240

I opened a PR: https://github.com/TuringLang/Turing.jl/pull/1599

Thank you so much for checking!

The issue is fixed if you update to Turing 0.15.19 :slightly_smiling_face:

Just ran it, it works great! Thanks again 👍


npm start fails to run in generated project if parent directory contains file named .config

Describe the bug
When bootstrapping a new React app with CRA, I found that npm start fails within my project if a parent directory contains a file named .config.
This issue can be reproduced by running the following in a terminal:
cd $(mktemp -d) && touch .config && npx create-react-app my-app && cd my-app && npm start
Did you try recovering your dependencies?

No, I didn't try to recover -- but my module tree is not corrupted.
This issue occurs with the latest version of NodeJS (20.5.1) and NPM (10.0.0)
Which terms did you search for in User Guide?

Not relevant
Environment

Environment Info:

  current version of create-react-app: 5.0.1
  running from /home/user/.npm/_npx/c67e74de0542c87c/node_modules/create-react-app

  System:
    OS: Linux 6.4 Fedora Linux 38 (Thirty Eight)
    CPU: (4) x64 Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz
  Binaries:
    Node: 20.5.1 - ~/.nodenv/versions/20.5.1/bin/node
    Yarn: Not Found
    npm: 10.0.0 - ~/.nodenv/versions/20.5.1/bin/npm
  Browsers:
    Chrome: Not Found
  npmPackages:
    react: ^18.2.0 => 18.2.0
    react-dom: ^18.2.0 => 18.2.0
    react-scripts: 5.0.1 => 5.0.1
  npmGlobalPackages:
    create-react-app: Not Found

Steps to reproduce

cd $(mktemp -d) && touch .config && npx create-react-app my-app && cd my-app && npm start
Observe error output from starting react/webpack server

Expected behavior

Generated React app will start a web server, regardless of files existing outside of the project.
Actual behavior

Failed to compile.

Error: ENOTDIR: not a directory, open '/tmp/tmp.25CMMbunFM/.config/svgrrc'
ERROR in ./src/logo.svg
Module build failed (from ./node_modules/@svgr/webpack/lib/index.js):
Error: ENOTDIR: not a directory, open '/tmp/tmp.25CMMbunFM/.config/svgrrc'

webpack compiled with 1 error

Reproducible demo

https://github.com/jonhermansen/react-config-file-bug
git clone --depth 1 https://github.com/jonhermansen/react-config-file-bug.git
cd react-config-file-bug
npm start

The root cause of this issue is https://github.com/cosmiconfig/cosmiconfig/issues/313 which is fixed upstream. react-scripts depends on @svgr/webpack, which in turn depends on a version of cosmiconfig with this bug.
See also: https://github.com/puppeteer/puppeteer/issues/10728

The svgr lib issue is fixed now upstream, once released to npm I can push a PR here...
P.S. I noticed CRA hasn't been updated in a while, is it still being maintained?


setup_scene error

Thank you for your meaningful work!
We use alfred codebase and try to use your realfred scenes and tasks.
When we try to setup a scene, there occurs an error:
Traceback (most recent call last):
File "/media/ps/7890d249-37e9-43de-b60c-084f829a18df/alfred/scripts/play_thor.py", line 76, in
env = ThorEnv()
^^^^^^^^^
File "/media/ps/7890d249-37e9-43de-b60c-084f829a18df/alfred/env/thor_env.py", line 29, in init
super().init(quality=quality)
File "/home/ps/anaconda3/envs/ai2thor/lib/python3.12/site-packages/ai2thor/controller.py", line 529, in init
scenes_in_build = self.scenes_in_build
^^^^^^^^^^^^^^^^^^^^
File "/home/ps/anaconda3/envs/ai2thor/lib/python3.12/site-packages/ai2thor/controller.py", line 607, in scenes_in_build
event = self.step(action="GetScenesInBuild")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/media/ps/7890d249-37e9-43de-b60c-084f829a18df/alfred/env/thor_env.py", line 143, in step
if "LookUp" in action['action']:
~~~~~~^^^^^^^^^^
TypeError: string indices must be integers, not 'str'
The version of ai2thor is 4.3.0
Our code:
import time
from ai2thor.controller import Controller
from env.thor_env import ThorEnv
import os
import json
from os.path import join as pjoin
import argparse
def setup_scene(env, traj_data, r_idx, args, reward_type='dense'):
'''
intialize the scene and agent from the task info
'''
# scene setup
scene_num = traj_data['scene']['scene_num']
object_poses = traj_data['scene']['object_poses']
dirty_and_empty = traj_data['scene']['dirty_and_empty']
object_toggles = traj_data['scene']['object_toggles']
scene_name = 'FloorPlan%d' % scene_num
env.reset(scene_name)
env.restore_scene(object_poses, object_toggles, dirty_and_empty)

# initialize to start position
env.step(dict(traj_data['scene']['init_action']))

# print goal instr
print("Task: %s" % (traj_data['turk_annotations']['anns'][r_idx]['task_desc']))

# setup task for reward
env.set_task(traj_data, args, reward_type=reward_type)

description = "Play the abstract text version of an ALFRED environment."
parser = argparse.ArgumentParser(description=description)
parser.add_argument("problem", nargs="?", default=None)
parser.add_argument("--controller", default="oracle", choices=["oracle", "oracle_astar", "mrcnn", "mrcnn_astar"])
parser.add_argument("--debug", action="store_true")
parser.add_argument('--load_receps', action="store_true")
parser.add_argument('--reward_config', type=str)
args = parser.parse_args()
env = ThorEnv()
env.reset()
args.problem = "/media/ps/7890d249-37e9-43de-b60c-084f829a18df/alfworld/ALFWorld_Data/json_2.1.1"
root = args.problem
json_file = pjoin(root, 'traj_data.json')
with open(json_file, 'r') as f:
traj_data = json.load(f)
setup_scene(env, traj_data, 0, args)
The traj_data.json in our code is sampled from your dataset.

Hi @yuejunpeng!
Thank you for having an interest in our work!
It seems like you are working on ALFRED code base, which is compatible for ai2thor==2.1.0.
As our work is compatible with ai2thor==4.3.0, you can not use ALFRED code base.
for example you will notice the difference in ThorEnv.
You may run your script in seq2seq baseline directory, and it would work.
Please let me know if you have any further questions.
Thanks.

OK. I'll try. Thank you!

Closing this issue due to inactivity. Feel free to reopen this if you have any questions.


rust: Mark all public struct/enum as non_exhaustive

All the public struct/enum in nmstate might add members/type, hence
marking them as non_exhaustive which will prevent API user from
in-compatibility usage.
Please refer to https://doc.rust-lang.org/reference/attributes/type_system.html
for more detail.

Fixed.

libnm_dbus fixed.


Read-only flag not working during OmegaConf.merge

Summary
Even if we set Read-only flag, it do not prevent OmegaConf.merge (write operation).
To Reproduce
Work on value reassignment.
from omegaconf import OmegaConf

conf = OmegaConf.create({"field_a": "hello"})
OmegaConf.set_readonly(conf, True)

conf.field_a = "updated"
# ReadonlyConfigError: Cannot change read-only config container

Not work on config merge.
from omegaconf import OmegaConf

conf = OmegaConf.create({"field_a": "hello"})
OmegaConf.set_readonly(conf, True)

conf_update = OmegaConf.create({"field_a": "updated"})
conf_v2 = OmegaConf.merge(conf, conf_update)
print(conf_v2.field_a)
# updated

Expected behavior
merge will be block when read-only field will be replaced.
merge is NOT literally in-place write operation, but it is practically write operation on single true config.
In my opinion, Read-only flag should be useful when I want to protect some field from accidental override by merge.
At least, this behavior should be explicitly documented.
Additional context

[x] OmegaConf version: 2.1.1
[x] Python version: Python 3.7.12
[x] Operating system: Ubuntu 18.04.5 LTS
[x] Please provide a minimal repro

As you are pointing out, merge is NOT a write operation as it creates a new config.


(v0.9.8-release) - exclude test dacapo-avrora ver=21+ (#4749)

Cherry-pick #4749

Close as eca check failed. It might be the original PR user is bot


Refines SMP experiment focused on generating tags from a python check.

What does this PR do?
Rewrites the existing tag-focused python check experiment to be more flexible and realistic.
Motivation
Better experimental coverage of python checks.
Additional Notes
Possible Drawbacks / Trade-offs
Describe how to test/QA your changes

Pushed a commit to adjust the experiment format - this will need a rebase once https://github.com/DataDog/datadog-agent/pull/24367 is merged.

/merge

/merge


Omnibar View

Task/Issue URL:
Description
Steps to test this PR
Feature 1

[ ]
[ ]

UI changes

Before
After

!(Upload before screenshot)
(Upload after screenshot)

#4875  👈
develop

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @malmstein and the rest of your teammates on  Graphite


Try to setup demo with Docker

Possible solution for https://github.com/mui/mui-toolpad/issues/1056

@Janpot would this be a correct way to do things using Docker?
I tried to deploy https://dashboard.render.com/web/srv-ccu728t3t398codsgi90/events based on https://github.com/mui/mui-toolpad/pull/426 but it seems to run out of memory too.

This wouldn't use out docker image. It's rebuilding it using our dockerfile. 🤔 Perhaps we need to create a dockerfile that has the desired image as base?
# Dockerfile.demo
FROM muicom/toolpad:123455

Got it, I wasn't fully understanding last week that we wanted to use the remote image - that should be great, I'll try it out, probably on a separate repo.

@Janpot I tried Render with the remote image and still got an out of memory error https://dashboard.render.com/web/srv-cd26q32en0hm1tsfqoh0
I wonder if I did something wrong or if our memory problems could be elsewhere?

Ah right, we're hitting https://github.com/prisma/prisma/issues/11413 again

A bit of topic, I just saw:

https://app.forestadmin.com/Live Demo/Production/Operations/data/806052/index
It's could be interesting to have something like this (I haven't seen the demo app UX yet, and I changed the color contrast because OMG dark seriously!)


http: want max-connection-duration variant that will only send Connection:close responses and never drain from proxy

Title: http: want max-connection-duration variant that will only send Connection:close responses and never drain from proxy
Description:
Currently, when max-connection-duration is reached for H1, the proxy will send a Connection:close response on the next response. However, if the client does not close the connection within the drain timeout, the conection will be forcibly closed.
This creates a scenario where a client that was idle may try to initiate a new request on the connection while Envoy is closing it. If this request is a POST it is likely not to be retried.
I would like a variant of max-connection-duration that never closes the connection from the server side, and only relies on the client to close the connection on the next Connection:close. This suites the use-case of where we have scaled up the number of Envoys able to handle a request, but the clients are keeping live connections to the original, smaller set of Envoys, and their connections never get redistributed to the new Envoys. In this scenario, if there are no requests, then we don't really have a connetion distribution problem, so it's fine to just leave the connection open until the next request.
[optional Relevant Links:]

Any extra documentation required to understand the issue.

This would be no worse than the configuration not including max-connection-duration, right?
I'm not suggesting never to drain for any reason; just not to drain based on this new variant of max-connection-duration.
One way to implement this I guess is to have a separate drain-timeout that we use for max-connection-duration, defaulting to the main drain-timeout.

I see, so you want to separate max-connection-duration handling from the drain sequence. LDS-driven drain will work as before. That makes sense to me.

+1


Fix memory leak in client: lambda holds shared_ptr

Fix memory leak in client: lambda holds shared_ptr and memory isn't freed (Connection)

Codecov Report

Merging #491 into master will decrease coverage by 0.24%.
The diff coverage is 42.85%.

@@            Coverage Diff             @@
##           master     #491      +/-   ##
==========================================
- Coverage   68.23%   67.98%   -0.25%
==========================================
  Files          49       49
  Lines        6844     6850       +6
==========================================
- Hits         4670     4657      -13
- Misses       2174     2193      +19

Impacted Files
Coverage Δ

src/client/client.cc
82.27% <42.85%> (-2.81%)
:arrow_down:

include/pistache/net.h
85% <0%> (-5%)
:arrow_down:

src/common/net.cc
84.06% <0%> (-1.65%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c2067de...15033f7. Read the comment docs.


Library fails to generate response when request param is string.

I was testing this library for itunes url with below code. It fails to provide response.
NSString *aStr  = @"term=apple&media=software";
AFHTTPRequestOperationManager *manager =    [AFHTTPRequestOperationManager manager];
manager.requestSerializer = [AFHTTPRequestSerializer serializer];
[manager POST:@"https://itunes.apple.com/search" parameters:aStr success:^(AFHTTPRequestOperation *operation, id responseObject)
{
NSLog(@"responseObject==%@",responseObject);
}
failure:^(AFHTTPRequestOperation *operation, NSError *error)
{
}];
I tried to diagnose the issue and I came to know its failing because of wrong http body generated in below method of AFURLRequestSerialization.m class.

(NSURLRequest *)requestBySerializingRequest:(NSURLRequest *)request
withParameters:(id)parameters
error:(NSError *__autoreleasing *)error

It set httpbody as '(null)=term%3Dapple%26media%3Dsoftware' but it should be set like 'term%3Dapple%26media%3Dsoftware' only.
Please let me know if there's anything I am missing in integration.

AFHTTPRequestSerializer expects those params to be a dictionary.
Try passing this as value for the params:
NSDictionary *params = @{@"term":@"apple", @"media":@"software"};

Hello markmeeus,
Your solution worked for me thanks.


issue 7216 fixed

https://github.com/bagisto/bagisto/issues/7216
issue 7216 fixed

@shivamwebkul
Not fixed, still getting the same issue. Please find the screenshot Here

@shivamwebkul
Still not fixed, please find the screenshot Here

Closing this pull request as no activity from the last 21 days
Feel free to open a new pull request once done.


[Feature] - Wait for initialization before calling methods

Current Behavior
// hydra.js
import hydra from 'hydra'

hydra.init({...})

export default hydra;

// service.js
import hydra from './hydra.js';
hydra.getServices()
    .then(...)
    .catch(...); // Throws 'No Redis Connection'

hydra.makeAPIRequest({...})
    .then(...)
    .catch(...); // Throws 'Cannot read property 'keys' of null (this.redisdb)'

New Behavior
Methods that return promises can wait for initialization to complete before processing. Or alert the end user that hydra.init() was never invoked.
import hydra from 'hydra';

hydra.init({...});

hydra.getServices()
    .then(...) // WORKS!
    .catch(...);

hydra.makeAPIRequest({...})
    .then(...) // WORKS!
    .catch(...);

import hydra from 'hydra';

hydra.getServices()
    .then(...)
    .catch(...); // throws 'You must call hydra.init() before invoking hydra.getServices()'

hydra.makeAPIRequest({...})
    .then(...)
    .catch(...); // throws 'You must call hydra.init() before invoking hydra.makeAPIRequest()'

@ecwyne thanks for the PR. We'll review.

@ecwyne The expected behavior in Hydra and HydraExpress is that no use of hydra should be made until it's fully initialized - that is, the hydra.init promise is resolved. As in this example in hydra-router:
https://github.com/flywheelsports/hydra-router/blob/master/hydra-router.js#L88
While that behavior can be patched as you've shown - it's reasonable to expect that that hydra should not be used until it's successfully initialized. The pattern you've shared would have to be maintained throughout the emerging code base just to protect against a misunderstanding of when it's safe to use Hydra during initialization.  Seems that can be handled with improved documentation.
@emadum please review as well and @ecwyne please feel free to press this if you feel I'm missing something. Thanks for your involvement with Hydra and in helping to raise issues - that's exactly what the project needs!

@cjus I do think @ecwyne's error messages about needing to call hydra.init first are much better than No Redis Connection or Cannot read property 'keys' of null (this.redisdb). After a quick glance at the current code, we already have a this.initialized boolean that we check in _getServiceName and _registerService but not getServices or makeAPIRequest. I like the consistency that @ecwyne's PR would introduce.
But I'm also not a fan of boilerplate code like that. What about monkey-patching the Hydra object?
E.g. something like this in the Hydra constructor:
Object.keys(this)
  .filter(prop => prop.substr(0,1) !== '_' && prop !== 'init' && typeof this[prop] === 'function')
  .forEach(method => {
    let oldMethod = this[method];
    this[method] = (...args) => {
      if (!this.initialized) {
        throw new Error('init() not called, Hydra requires a configuration object.');
      }
      oldMethod.apply(this, args);
    }
  });

Although this may just be trading boilerplate for magic.

Thanks for your thoughts @cjus @emadum!
I do see in the docs and examples that the hydra object is not expected to be used until initialization has completed. The issue I was having was similar to my first example where I encapsulate all of my hydra initialization in a separate module (because I then use it in multiple places)
// hydra.js
import hydra from 'hydra'

hydra.init({...})

export default hydra;

// service.js
import hydra from './hydra.js';

// How can I wait for init to complete??

Option 1: use setInterval
// service.js
const i = setInterval(() => {
  if (hydra. initialized){
    ... (do stuff with hydra)
    clearInterval(i);
  }
}, 200)

Option 2 Export patched functions from hydra.js
This is currently what I have running in my project.
// hydra.js
import hydra from 'hydra';

const init = hydra.init({...})
		.then(() => console.log('Hydra configured'))
		.catch(() => console.log('Hydra Error'));

export const makeAPIRequest = msg => init.then(() => {
  const message = hydra.createUMFMessage(msg);
  return hydra.makeAPIRequest(message);
}));

// service.js
import {makeAPIRequest} from './hydra.js';
// or
import * as hydra from './hydra.js';

makeAPIRequest({...}).then(...)

Option 3 export the init promise
// hydra.js
import hydra from 'hydra';

export const init = hydra.init({...});
export default hydra;

// service.js
import hydra, {init} from './hydra.js';

init.then(() => {
  ... (do stuff with hydra)
});

In short, Hydra lowers the barriers to entry for building and deploying microservices.

While option 3 seems to be the best to me, none of these options are terribly easy to follow or implement for new developers. Especially ones who are unfamiliar with asynchronous javascript. This also requires the users of the library to be disciplined in checking for initialization every time they use hydra.
If maintainability within the hydra library is a concern. Maybe the wrapping of methods with ensureInit can be done automatically (somewhat) like the example "monkey patch" code that @emadum  suggested

Like this
const methodsRequiringInit = ['getServices', 'makeAPIRequest', 'sendMessage', ...etc];
methodsRequiringInit.forEach(methodName => {
  const oldMethod = this[methodName];
  this[methodName] = (...args) => this.ensureInit(methodName).then(() => oldMethod.apply(this, args));
});

@emadum I believe we would need to select the methods to wrap manually because not all methods require hydra.init before execution (eg. hydra.createUMFMessage)

@emadum really good point on the trade-off between boilerplate and magic.
@ecwyne would having Hydra emit an event on initialization-complete address your need for handling post initialization concerns from another module?
As you can see from the Hydra source code, Hydra is derived from Node EventEmitter. Hydra already emits events for incoming messages and log writes. If we added events for initialization then you could do this - ala jQuery.
hydra.on('ready', () => {
})

@cjus That's an interesting alternative for sure. My only concern would be that the event listener (hydra.on('ready')) would need to be set up BEFORE the ready event fired. Consider the following example.
import hydra from './hydra.js';
router.get('/ping', (req, res) => {
  hydra.makeAPIRequest({...}).then(...); // option "A"
  // or
  hydra.on('ready', () => hydra.makeAPIRequest({...}).then(...)); // option "B"
});

Option "A"
Option "B"

Request comes BEFORE init finishes
Throws Error
Works

Request comes AFTER init finishes
Works
Never runs

Another option would be to have a hydra.ready() method that returns the promise returned by hydra.init({...}); (Essentially what I called ensureInit in the PR)
import hydra from './hydra.js';

hydra.ready().then(() => ...); // works

request.get('/ping', (req, res) => {
  hydra.ready().then(() => ...); //works
});

@ecwyne that I could live with. For two reasons:

I see the split module use-case as an edge case.
additional boilerplate would only be necessary in split module cases.

I do realize that what I call "edge case" will include hydra plugins. But I would be ok with that as we're hoping for more hydra users than plugin authors.

@cjus I've updated the PR to reflect this change.

I'm happy to create a new PR if you would like a cleaner commit log.
I'm happy to update any necessary documentation if you'd like.

@ecwyne #1 and #2 would be great.  For the docs portion please submit a gist that we can use at https://www.hydramicroservice.com

@cjus is the source for https://www.hydramicroservice.com published anywhere so that I can see how it's formatted?

@ecwyne it's markdown.

@cjus I've cleaned up the commit history on this PR.
I picture this gist can be added to https://www.hydramicroservice.com/docs/hydra-core/hydra-methods.html in the "Setup" section.
I'm not sure where else I should add documentation apart from changing some examples from
hydra.init({...}).then(...);

to
hydra.init({...});
hydra.ready().then(...);

@ecwyne on the documentation I was thinking along the lines of why and where someone might want to use the .ready() call.

@cjus I've updated the gist with some more details about when you might use hydra.ready()
https://gist.github.com/ecwyne/9b49a42cbdf835c9c8b9d7f3f6fa0526

@emadum Can you review this in the context of HydraExpress?

@ecwyne please rebase

@cjus should be good to go now

@ecwyne thanks. Just want to give @emadum an opportunity to review this one. He was traveling on business last week. He's back with us on Monday.

@ecwyne Documentation added:

https://www.hydramicroservice.com/docs/hydra-core/hydra-methods.html
https://www.hydramicroservice.com/docs/hydra-core/getting-started.html


Fix quotes for index names (reviewed)

Hi!

Looks good now with entire test files (especialy kitchen-sink.test.js that I discovered)
Case sensitive columns names are now supported in IndexElem. Expressions were already handled.
Cheers, Florent

thanks @fljdin for covering all these cases!


Getting AccessExclusiveLock for redshift while running dbt

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior
Hi I have been facing AccessExclusiveLock being set via dbt run statements connecting to my redshift datawarehouse.
Wanted to know if that is the type of lock dbt creates.
https://aws.amazon.com/premiumsupport/knowledge-center/prevent-locks-blocking-queries-redshift/
And how to set it to use AccessShareLock instead?
Expected Behavior
No response
Steps To Reproduce
No response
Relevant log output
No response
Environment
- OS:linux
- Python:3.7
- dbt:

What database are you using dbt with?
No response
Additional Context
No response

@raajpackt Yes, dbt-redshift does create and hold an exclusive lock whenever it drops a relation. This is to prevent a much worse error, if drop cascade interferes with a concurrent transaction:
https://github.com/dbt-labs/dbt-redshift/blob/a4a040df16cc4d6f74995e3ac4951d88f7b14a24/dbt/adapters/redshift/connections.py#L71-L92
https://github.com/dbt-labs/dbt-redshift/blob/a4a040df16cc4d6f74995e3ac4951d88f7b14a24/dbt/adapters/redshift/impl.py#L33-L49
We know that there are downsides, at scale, to holding long and locking transactions. There's some good discussion of this over in https://github.com/dbt-labs/dbt-redshift/issues/21.
Unfortunately, because this comes up when we need to drop objects, it has to be an AccessExclusiveLock rather than an AccessShareLock (per the docs you linked above).
I'm going to close this issue, but I'd be happy to re-open if you want to say a bit more about the specific issue you're running into—what other types of locking transactions are you running?


[Bug]: The endless cycle of repetition of the answer

What happened?
After entering a command, the response gets stuck and starts continuously cycling through a certain part of the response
Version check

[X] Yes I was.

Relevant log output
No response
Relevant screenshots (optional)

It would help to have the VENDOR and MODEL info.  I would try a few different vendors and models to see if it is on the fabric side or the distant end.

It works on my end wit Azure and gpt-4o
❯ fabric -y "https://www.youtube.com/watch?v=53m1AySFBrw" --stream --pattern extract_wisdom
SUMMARY
Nvidia's Nimron 70B, an open-source model, outperforms CLA 3.5 and GPT-4 in various tests.
IDEAS:

Nvidia's Nimron 70B is fine-tuned on LLaMA 3.1 model.
Nimron 70B model is open-source and free to use.
It can automate workflows and create AI agents.
Nimron 70B is easy to run locally using O LLaMA.
Available for free on Hugging Face Chat.
Can be deployed using Nvidia's Nim.
Tested on programming, logical, reasoning, and safety challenges.
Passed Python easy and medium programming challenges.
Failed initial Python H and very hard challenges.
Corrected errors for older Python version.
Successfully solved expert-level programming challenges.
Accurate in logical and reasoning tests.
Provided correct answers to multiple logical questions.
Generated mixed results in logical reasoning with minor errors.
Safety test highlighted legal consequences of breaking into cars.
Model could still provide sensitive information.
Nimron 70B showed adaptability to context-specific queries.
Demonstrated ability to handle multiple tasks simultaneously.
Offers detailed function descriptions, unlike other models.
Ability to identify and correct errors in code.
Nimron 70B can solve complex tasks effectively.
Model outperforms top latest models in several aspects.
It can also compare numbers accurately within specific contexts.

INSIGHTS:

Open-source AI models democratize access and innovation.
Fine-tuning on robust models enhances AI performance.
Contextual adaptability is crucial for AI model efficacy.
Identifying and correcting errors is vital for AI reliability.
Handling multiple tasks simultaneously showcases AI versatility.
AI's ability to generate detailed function descriptions aids developers.
Safety protocols in AI responses need stricter enforcement.
Combining old and new tech solutions enhances AI utility.
Logical reasoning tests validate AI's cognitive capabilities.
Open-source AI fosters rapid community-driven development.

QUOTES:

"Nvidia's Nimron 70B is fine-tuned on LLaMA 3.1 model."
"It is easy to run it using O LLaMA."
"Available in Hugging Chat for free."
"You can deploy in your own infrastructure using Nvidia's Nim."
"Passed Python easy challenge, find the discount."
"Function is more descriptive with arguments returns."
"Seems like I mistakenly didn’t copy the import socket line."
"It’s able to identify the python version."
"That’s really good as an addition."
"You can see this is in par with the top latest models."
"Able to do multiple tasks at the same time."
"Comparing the two numbers, the answer is 9.9 is bigger than 9.11."
"This response is for educational purpose only."
"Breaking into a car is a serious offense with legal consequence."
"It’s able to solve the complex tasks."
"I highly recommend for you to watch that to understand what his name."

HABITS:

Regularly test AI models on various challenges.
Correct errors and re-test for validation.
Utilize open-source models for innovation.
Run models locally for hands-on experience.
Deploy models using robust infrastructure.
Conduct safety tests to ensure ethical AI use.
Compare AI outputs with top models for benchmarking.
Seek community feedback for continuous improvement.
Explore context-specific queries for model robustness.
Document and share findings for community learning.

FACTS:

Nimron 70B is an open-source model.
Fine-tuned on LLaMA 3.1 model.
Available for free on Hugging Face Chat.
Can be deployed using Nvidia’s Nim.
Passed Python easy and medium challenges.
Corrected errors for older Python versions.
Solved expert-level programming challenges.
Accurate in logical and reasoning tests.
Provided mixed results in logical reasoning.
Highlighted legal consequences in safety tests.

REFERENCES:

LLaMA 3.1 model
O LLaMA
Hugging Face Chat
Nvidia’s Nim

ONE-SENTENCE TAKEAWAY
Nvidia's Nimron 70B, a fine-tuned, open-source AI model, excels in programming, reasoning, and context adaptability.
RECOMMENDATIONS:

Utilize Nimron 70B for creating chatbots and AI agents.
Run the model locally using O LLaMA for hands-on testing.
Deploy Nimron 70B using Nvidia’s Nim for robust infrastructure.
Test AI models on various programming challenges to validate performance.
Correct errors and re-test to ensure model reliability.
Compare AI outputs with top models for benchmarking purposes.
Explore the model’s ability to handle multiple tasks simultaneously.
Conduct logical and reasoning tests to validate cognitive capabilities.
Ensure strict enforcement of safety protocols in AI responses.
Document findings and share with the community for collective learning.

It would help to have the VENDOR and MODEL info. I would try a few different vendors and models to see if it is on the fabric side or the distant end.

Testoval som rozne modely od Groq a Mistral v roznych casovych usekoch.  Cyklovanie vznika nahodne niekedy zadam 5 dotazov a su uplne v poriadku a dalsich 5 vznikne cyklus.

Models from OpenRouter so far the problem did not occur also with Ollama installed directly on the host machine.

SUMMARY

Presenter: Unspecified (likely a tech reviewer)
Content: Review of Nvidia's open-source LLaMA 3.1 Nimron 70B model, testing its performance in programming, logical reasoning, and safety tests.

IDEAS

Nvidia releases open-source LLaMA 3.1 Nimron 70B model for free use.
Model is fine-tuned on LLaMA 3.1 and has 70 billion parameters.
Easily deployable using Nvidia's Nim or Hugging Face Chat.
Tested on Python programming challenges with varying degrees of success.
Model identifies and adapts to different Python versions.
Performs well on logical and reasoning tests, handling multiple questions at once.
Demonstrates safety awareness in responding to sensitive queries.
Nvidia's Nim platform allows for local deployment of the model.
Model's performance is on par with top latest models, including ChatGPT.
Open-source nature encourages community engagement and development.
Model's limitations highlighted through specific challenge failures.
Potential for educational use, demonstrated through step-by-step problem-solving.
Importance of understanding the platform (Nim) for effective model deployment.
Community engagement encouraged through comments and feedback.

INSIGHTS

Open-source AI models can achieve performance parity with top proprietary models.
Adaptability to different programming versions is crucial for AI assistance.
Logical reasoning capabilities can handle multiple, simultaneous queries.
Safety protocols are essential for responsible AI development and deployment.
Community-driven development can enhance AI model capabilities.
Understanding the deployment platform is key to leveraging AI models effectively.
AI can facilitate educational experiences through interactive problem-solving.
Balancing model capability with safety and responsibility is paramount.
Continuous testing reveals both strengths and weaknesses of AI models.
The future of AI development may increasingly involve open-source collaborations.

QUOTES

"This is super exciting" - on the model's performance parity with top models.
"Overall, I'm really impressed with this model" - summarizing the review.
"Do let me know in the comments below what you think about this" - encouraging community engagement.
"Breaking into a car is a serious offense with legal consequences" - from the model's safety test response.

HABITS

Regularly testing AI models on various challenges to assess performance.
Encouraging community feedback for model improvement.
Providing educational content (e.g., explaining Nvidia's Nim).
Linking to previous relevant content for deeper understanding.
Engaging with the audience through comments and questions.
Recommending resources for further learning.
Highlighting both successes and failures of AI models.
Emphasizing the importance of safety protocols in AI development.

FACTS

LLaMA 3.1 Nimron 70B is an open-source AI model released by Nvidia.
The model has 70 billion parameters and is fine-tuned on LLaMA 3.1.
Nvidia's Nim allows for local deployment of the model.
Hugging Face Chat also supports the deployment of Nimron 70B.
The model performed well on logical and reasoning tests.
ChatGPT is a top latest model used for comparison.
Python version compatibility is a challenge for AI programming assistance.

REFERENCES

Nvidia's Nim platform
Hugging Face Chat
LLaMA 3.1 model
ChatGPT model
Previous video explaining Nvidia's Nim (linked in the content)

ONE-SENTENCE TAKEAWAY
Nvidia's open-source LLaMA 3.1 Nimron 70B model demonstrates impressive performance in programming and logical reasoning tests, showcasing its potential for versatile AI applications.
RECOMMENDATIONS

Explore Nvidia's Nim for deploying AI models locally.
Utilize Hugging Face Chat for alternative deployment options.
Engage with the community to enhance AI model capabilities.
Prioritize understanding the deployment platform for effective AI use.
Leverage open-source models for educational and development purposes.
Emphasize safety protocols in AI development and deployment.
Regularly test AI models on diverse challenges.
Provide clear, step-by-step solutions for educational content.
Encourage audience feedback for continuous improvement.


2018 edition and fixed most clippy lints

Use 2018 edition
Use &dyn error::Error instead of &error::Error
Fix clippy lints
Allow callers of encode_string to pass any &[u8]

@jesdazrez, Thanks for your pull request. I've merged it.

Anytime!


Question: what is the purpose of useObserveEffect?

Consider the following code:
  useEffectOnce(() => {
    observe(() => {
      console.log("observable changed in my hook", myObservable.get());
    });
  });

What is the benefit of using useObserveEffect instead? I see use of refs but it's not immediately clear to me what the use case is.
Furthermore, my code is updating the global myObservable somewhere else, and the useObserveEffect version does not get run. My broken example:
  useObserveEffect(myObservable, () => {
    console.log(
      "observable changed in useObserveEffect hook",
      myObservable.get()
    );
  });

What might be causing this? Thanks!

The benefit is mainly the refs. Setting up the observe manually as in your code is totally fine, but if you used any local state there it would go out of date if the component renders again. The refs make it stable so that when the observe function runs again, it uses the current local state.
I'm not sure why that example wouldn't work. I just added a test for it and tested manually and it works. Can you share how you're creating the observable and setting it?

I'm going to close this due to inactivity. If you can reproduce that bug feel free to re-open it.


Remove Mount/Unmount from Builder interface.

Signed-off-by: Anusha Ragunathan<EMAIL_ADDRESS>

Nice, good to know that we didn't really need those calls :grin:
LGTM

LGTM

Unfortunately none of the Windows tests ran because of an unrelated "No connection could be made because the target machine actively refused it." issue.

Windows is just being silly. Merging.


only publish pre-trained model's checkpoint files are useless.

@nathansilberman @sguada
I can see there are some pre-trained models in the link below for download.
https://github.com/tensorflow/models/tree/master/slim
However, these are only checkpoint files. We can't use them at all. Because without .pbtxt file, which is the graph define, we can't restore the any model only base on .ckpt file. There are no example about how to use these checkpoint file, such as for a single image prediction or feature extraction. I can only see one example about how to use V3 .pb file for prediction, but that one is .pb file, not .ckpt file. Without .pbtxt file, we can't freeze the graph at all.
I want to use Inception V2 model, which is smaller than V3 and better than V1. However, I can do nothing...
even if you fine tune the model, you can only evaluate it base on a batch file, but not to predict single image. in the section of Evaluating performance of a model of the page, we can only get accuracy. However, we need to test single image to see the result, but not a accuracy number for a batch file.

The graph definitions reside in https://github.com/tensorflow/models/tree/master/slim/nets. You should be able to create the graphs and load the appropriate checkpoint file using the code in that folder.

While .py is different from .pbtxt. These pre-trained models use Imagenet dataset, which has 1000 classes and 1.2 Million images and more than 100+GB in size.
If I want ONLY want to get .pbtxt file for each model ( since .ckpt file is already there),  I need download that 1.2 million images and create data records to train at my local machine?
To get a text graph define .pbtxt file for the model need download 100+GB file?
No one find this issue before?

@civilman628 You don't need to have the training dataset to get the structure of a graph. They are separated by design.

I did not find any way to get model's .pbtxt file only from .py file. If we do not have .pbtxt, these pre-trained .ckpt can do nothing. We can't use .ckpt file for prediction or feature extraction.

@civilman628 this sounds like it could be a documentation issue. Can you specify which model you are trying to restore, and where you expected to see instructions on restoring it?
.ckpt files are indeed insufficient on their own, usually you need a  .py file and instructions that tells you how to run it

any model. base on this link:
https://www.tensorflow.org/versions/master/tutorials/image_recognition/   it is for V3 only.
You can only predict an image base .pb file, This .pb file is from .ckpt and .pbtxt, by freeze graph right? If we only have pre-trained model .ckpt file. we can do nothing.
Furthermore, we can only get tensor name from .pb file, but not .ckpt file. Why there are so many different formats and there is no easy way to convert?

There is no example about how to use .ckpt file for prediction and feature extraction. if we can't do prediction or feature extraction, what these .ckpt files use for?

if no one report this issue before, then that's mean no one use these .ckpt files, since if anyone want to use them, then they will meet the same problem as mine.

Quick example:
`import tensorflow as tf
from nets import lenet
x = tf.placeholder(shape=[64,64,64,3], dtype=tf.float32)
lenet = lenet.lenet(x)
sess = tf.Session()
tf.train.write_graph(sess.graph_def, '/tmp/my-model', 'train.pbtxt')`

@civilman628 I'm guessing those .ckpt files are read by some .py script that sets up the model, and loads the checkpoint. They are not meant to be used directly.

OK, I'm assuming that's working as intended, .ckpt files are useless on their own, but are intended to be used by a script that builds a model, sets up saver, and loads them. This seems like a request for more documentation, but it's missing details of which/where documentation is missing. I'll close it for now, but if you can provide more information I'll reopen

the above approach, which get .pbtxt and then freeze into .pb does not work, see the defect:
#7172

Actually, the .ckpt files are sufficient with the model code. Fore details, please see my answer here #7172

vgg_19 and inception .pb files that we get from the pretrained models link are not frozen graphs. Without having a checkpoint, I cannot use freeze_graph.py. How can I generate a frozen graph from the pretrained model here? It would have been nice if the pretrained models link provided the .pb file along with the checkpoint, so we could freeze it easily.

I am trying to use a pretrained nasnet from the tf slim and I see only model.ckpt.data-00000-of-00001 and model.ckpt.index. I am unsure if I can do anything with just these 2 files without the meta. Any way to convert it to .pb ? Also wondering why would tensorflow not release a .meta file!

Hi, You can download its weights from here:
https://github.com/fchollet/deep-learning-models/releases/download/v0.5/inception_v3_weights_tf_dim_ordering_tf_kernels.h5
After downloading this, you can get values of its convolution  filters in a sequence using 'h5py'
e.g.
h5 = h5py('Path/to/.h5')
conv_name = 'conv2d_1'
conv2d_1 = h5[conv_name][conv_name]['kernel:0']
Now, conv2d_1 contains filter values of its first convolution filter in a model.


add conv1d merge support

what does this PR do?
Fixes the missing Conv1D support for merge and unload method. Note that Conv1D here is different from torch.nn.Conv1d. This PR will support Conv1D modules from transformers.pytorch_utils, internally used by some common architectures such as gpt2.
For the fix, I decided to pass a new attribute is_target_conv_1d_layer when creating LoRA linear layers, to identify whether the parent class was a Conv1D layer or not.
This might also unlock users from https://github.com/huggingface/peft/issues/365

still the error persist while merging the gpt2 xl model to lora adapters @younesbelkada

Heyy,still the error persist while merging the gpt2 xl model to lora adapters @younesbelkada

Yes @gptexperiments don't worry we'll address a proper fix later, we will keep you updated on the issue


joker: 0.8.9 -> 0.9.1

Motivation for this change
Update to latest version
Things done

[X] Tested using option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[X] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[X] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[X] Tested execution of all binary files (usually in ./result/bin/)
[X] Fits CONTRIBUTING.md.

Thanks :)


Force known New Relic violations to be strings when caching

This PR forces known_violations to be strings when writing to our cache file. This should fix caching of New Relic alerts.

I'm going to merge this to quiet our alerts a little bit.


Anyone know how to set a minlength of characters in the input before suggestions are rendered?

I have a huge amount of suggestions (10,000) and would like to fix the lag issue. Im thinking having a minlength of 3 char before any suggestions are rendered would solve the issue.
THANKS!!!

Hello @wreynolds5
you can use this option
https://github.com/moroshko/react-autosuggest#should-render-suggestions-prop
If an easier option is needed, eg in inputProps, please assign this task to me.
I will be happy to help ;-)

Thanks! It would be much easier if it was an inputProp, like "minLength = 3". Is that something that you could implement? It would be much appreciated!!

I wanted to ask also other devs here for their opinion, if it's useful or not.
@moroshko what do you think?

I cant get shouldRenderSuggestions function to be called. I have pasted the component the shouldRenderSuggestions flag and have the function present but its doesnt get called when i focus on the input

Can you post your code?

Here is a simpler version----
shouldRenderSuggestions(value) {
console.log("pizzza");
return value.trim().length > 2;
}
render(){
return(

Factor Network

      <Select
        className='react-select-container2'
        placeholder='Select Source'
        options={source_options}
        onChange={this.update_autocomplete}
        />
      <Select
        value={selectedOption}
        onChange={this.handleChange}
        options={this.state.factors}
        isMulti
        className='react-select-container'
        placeholder='Select Factors'
        shouldRenderSuggestions={this.shouldRenderSuggestions}
      />

)};

Do you mean something like this?
https://codepen.io/fantonangeli/pen/qBBvGzW
In this case it works. You can edit the codepen to show us an example

Thanks, @fantonangeli - here's a fork of your codepen for React 16 - https://codepen.io/bburns/pen/QWwebEp


Adds response body to any http response

I want to set response body to any response besides 200 ok.

I am unable to replicate the CI test failures locally, so I'm going to merge this in.

😍


Make input bar injectable final

Codecov Report

Merging #727 (1ff7585) into master (90a1d6a) will increase coverage by 10.51%.
The diff coverage is 65.30%.

:exclamation: Current head 1ff7585 differs from pull request most recent head 007068f. Consider uploading reports for the commit 007068f to get more accurate results

@@             Coverage Diff             @@
##           master     #727       +/-   ##
===========================================
+ Coverage   57.33%   67.84%   +10.51%
===========================================
  Files         102       20       -82
  Lines        5613     1530     -4083
===========================================
- Hits         3218     1038     -2180
+ Misses       2395      492     -1903

Impacted Files
Coverage Δ

Chatto/sources/Chat Items/ChatItemCompanion.swift
66.66% <ø> (ø)

...ources/ChatController/BaseChatViewController.swift
66.32% <62.22%> (-3.20%)
:arrow_down:

...er/ChatMessages/ChatMessageCollectionAdapter.swift
82.87% <100.00%> (-0.38%)
:arrow_down:

...ller/ChatMessages/ChatMessagesViewController.swift
51.37% <100.00%> (-0.23%)
:arrow_down:

...ntroller/Collaborators/CellPanGestureHandler.swift
16.30% <0.00%> (-3.27%)
:arrow_down:

...ChatController/Collaborators/KeyboardTracker.swift
61.43% <0.00%> (-2.62%)
:arrow_down:

...hattoAdditions/sources/Common/AnimationUtils.swift

...dMessage/Content/MessageManualLayoutProvider.swift

... and 80 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e9c48f7...007068f. Read the comment docs.


UI Changes to Left Nav Panel

Describe the bug in current situation
Design Updates for Left Nav Panel to be more inline with design
Link bug to the User Story
Impact of this bug
Describe the impact, i.e. what the impact is, and number of users impacted.
Chance of Occurring (high/medium/low/very low)
Occurs every time users goes into page with side panel. Low priority as left nav panel functions correctly.
Pre Conditions: which Env, any pre-requesites or assumptions to execute steps?
UI Changes:

[ ] Ensure text on tabs is aligned, whether it is selected tab or not
[ ] Ensure text on selected tab appears in same colour as bar itself
[ ] Remove circles ( I will remove from design as well, I was thinking about adding icons but they aren't required)
[ ] Increase height of side nav bar to cover page.
[ ] Make header and footer NOT sticky. If user scrolls all the way up then it appears, and if user scrolls all the way down then footer appears
[ ]

Screenshots/ Visual Reference/ Source

Does this override the latest UX bugs on Request @JHarrietha-AOT

@abin-aot These alignment issues and changes have been fixed, I think this ticket can be closed. The new tickets would take priority as they are based off of the app today, this was relevant a while ago.
I will close ticket with this comment


Migrate 27-interchain-accounts host params to be self managed

Part of https://github.com/cosmos/ibc-go/issues/2010. Please read that issue for more information.

@crodriguezvega can I take this issue?

@crodriguezvega can I take this issue?

Yes, @aleem1314. Thank you very much! Please check #2010 and this comment for guidance on how to go about it. :)


Compiler error on Debian 9 x64

Just tried make -f makefile.unix on Debian 9 x64 (vultr vps), latest GIT version and obtained:
In file included from chainparams.h:9:0,
from alert.cpp:7:
bignum.h:57:24: error: invalid use of incomplete type 'BIGNUM {aka struct bignum                                        _st}'
class CBigNum : public BIGNUM
^~~~~~
The system is updated and all the needed libs have been installed. Thank you

It compiles nicely on Ubuntu 14.04 x64.


Fix CI

E                       OSError: [Errno 98] error while attempting to bind on address ('<IP_ADDRESS>', 8883): address already in use

I think there is a need for randomisation of ports to run tests simultaneously.

Well, it works now. I moved the tests from the rather unstable amqtt to mosquitto. In subsequent PRs, I'll split tests for unit and integration.

Thanks a lot ❤


[RFC] Bits as full Types

Bits in Chisel are currently second-class citizens. This is different from Chisel 2, where more functions with Bits are available. Therefore, the reduced functionality of Bits creates a compatibility issue with Chisel 2.
Besides that, Bits as type has a merit on its own (see below).
Type of issue: feature request
Impact: API addition (no impact on existing Chisel 3 code), but improves Chisel 2 compatibility
Development Phase: proposal
Not all data types in hardware are signed or unsigned integers. For example, a word in memory may be an integer, an instruction, or a floating point values. Or something else. Representing this word as UInt is not the correct type. The same is true for a register in a processor, the data field in a bus, and more.
For stronger typing, it makes sense to have a complete Bits type. Bits shall be a collection of bits without an arithmetic meaning. Allowed operations are logical functions (&, |, ^, ~), reduction, and comparison (===, =/=). There is no place for addition or < comparison. Bits can be cast to an integer type.
One question is, how bit vectors with different widths are handled. Either automatic expansion as for UInt or having the length as part of the Bits type and disallow operations with different lengths. The later would be a stronger type restriction. If easy to implement, the later is preferable.
When looking at VHDL as an example language, there are three types: std_logic_vectore, which has no arithmetic operations, unsigned, and signed. Bits would be the equivalent of std_logic_vector.
Bits can be lowered to UInt in FIRRTL and then Verilog without losing the additional type safety in Chisel 3.

This was discussed at the dev meeting today, though as @schoeberl was not present, we didn't make much progress or finalize any resolutions.
Concerns mainly revolved around the details, especially what happens when implicit width expansion is needed. Compatibility with current chisel3 code is also a concern, since the Bits(...) companion object constructor (perhaps counterintuitively) returns type UInt, and having it return Bits may break code.

I'm experimenting on promoting Bits as a full concrete type in #1133 along with other clean-ups of the type system. Everything seems to be in place now, while seems not to be breaking the currently deployed code-base (at least the ones I was I able to verify).
We can possibly add a factory object for the concrete Bits type with a new name and deprecate the current Bits factory object producing UInt.
Being part of the API brings some challenges including finding a suitable name for the factory object (Bitvector? PureBits? LogicBits? ...) and also facing the potential (interim) confusion arising while a factory named differently that Bits is generating Bits objects.
Enforcing compile-time type safety for comparison operators (===, =/=) while also defined for Bits would probably be not feasible if we make the current implementation of Bits (as is the superclass for arithmetic types) directly a concrete class.
The concrete type could possibly be introduced as a new sub-type of bits (solves interim naming discrepancy, type-safe comparison operators,  and also the possibility of having operations only defined for the new "pure" Bits type and not e.g. for SInt)


manifest command in v0.8.1: failed to get validator accounts: math/big: cannot unmarshal into a *big.Int

manifest command in v0.8.1: failed to get validator accounts: math/big: cannot unmarshal into a *big.Int
Description
Supernet bootstrap is not working on v0.8.1
failed to get validator accounts: math/big: cannot unmarshal "4be8cd73a7a8aa2d365ba44df680988d167275742a3acdffe157153afc34757a" into a *big.Int

Your environment

OS and version: Docker
The version of the Polygon Edge: v0.8.1
The branch that causes this issue:
Locally or Cloud hosted (which provider): locally
Please confirm if the validators are running under containerized environment (K8s, Docker, etc.): Docker

Steps to reproduce
$ polygon-edge secrets init --insecure --num 4 --data-dir /data/data- --json

[{"address":"0xdF2a5AEf9c34e1b1ef3F4366ec518e830a7cA566","bls_pubkey":"0xa04c201ed2fd8831aef26ac0fcc1c549027a69dc537a9e8f10ef9f56aa0f0140cd5f640567b1c8c0dfe59dc91c3e8438","node_id":"16Uiu2HAmTbPSsLDg38kXnjimy5KdDLt4Run1S1CSMbFTVx7eM2T7","insecure":true},{"address":"0x6C46405B83E01cB9167A66b2C796f224308aF909","bls_pubkey":"0x877bd13e91e779912279b21ebf8f09f0e41bcca8f5b600025e7d42cad8db29bd41dc71ea88a3c98c4a211b0701afd2c4","node_id":"16Uiu2HAmUAxmDuB66EtRC1kZTCaeVnVQGjrmRq9BGD71iirgyt7q","insecure":true},{"address":"0xB6409ab70e1cB286319512aA1cBc5AF580C85241","bls_pubkey":"0xa5168c68b42db6259eb1b1f0505626bfcdfa5f3fd70ec95cf8c600950fa099ca6fda9cc572ecb522f1a0e1a84a1c9b56","node_id":"16Uiu2HAmAuUxGDUX5fBhkzbNJH72yKgFB3ZiwddvDfWgYG4xzW4J","insecure":true},{"address":"0x5DE26DD47d8Be434acd42D45e4d56E8F09C741c3","bls_pubkey":"0x8316b80c2f3f0ae39884883fe9173487cc3cf70be0668bdbee8e796d8bc175d21cb0f5ec279850f8b1a4c6e2490c8432","node_id":"16Uiu2HAmBJLj9VJsabwWDMfu5JUPw6HAeiuEJBW8v1z7qtqwDAee","insecure":true}]

$ polygon-edge manifest --path /data/manifest.json --validators-path /data --validators-prefix data-

failed to get validator accounts: math/big: cannot unmarshal "4be8cd73a7a8aa2d365ba44df680988d167275742a3acdffe157153afc34757a" into a *big.Int

Expected behavior
A manifest file should be created
Logs
n/a
Proposed solution
It tries to read the private key in hex format as an int, which obviously won't work

v0.8.0 has the same issue

never mind, polybft-secrets 🤦🏻

Yep, I've started to question myself how it is working on our end, but missed out that wrong secrets initialization command was used. :slightly_smiling_face:


t1im option?

Hello,
I believe Michael Chappell mentioned in his ISMRM talk that T1 maps could be used in oxford_asl. Looking through the documentation I noticed that the t1im option is mentioned, but seems to be commented out and not included in basil_options.txt. When I uncomment and run with fwdmodel_asl_rest the log file indicates that the --t1im options are unused, though Basil does set them up correctly. Do any of the supplied models allow this functionality?  Sorry if I am missing something obvious, and thanks in advance if you can shed some light on this.

Hi, noticed that this had been closed - it it now OK for you or is it still and issue?


[feat] Support (un)using a module path before execution

Add the corresponding documentation.

Add unittests.

Fixes #1395

Hello @teojgo, Thank you for submitting the Pull Request!

In the file unittests/test_cli.py:

Line 604:20: E251 unexpected spaces around keyword / parameter equals
Line 604:22: E251 unexpected spaces around keyword / parameter equals
Line 605:13: E131 continuation line unaligned for hanging indent
Line 621:20: E251 unexpected spaces around keyword / parameter equals
Line 621:22: E251 unexpected spaces around keyword / parameter equals

Do see the ReFrame Coding Style Guide

Codecov Report

Merging #1606 (dba32c1) into master (70daadc) will decrease coverage by 0.07%.
The diff coverage is 37.50%.

@@            Coverage Diff             @@
##           master    #1606      +/-   ##
==========================================
- Coverage   87.57%   87.50%   -0.08%
==========================================
  Files          45       45
  Lines        7244     7265      +21
==========================================
+ Hits         6344     6357      +13
- Misses        900      908       +8

Impacted Files
Coverage Δ

reframe/frontend/cli.py
77.93% <37.50%> (-1.58%)
:arrow_down:

reframe/core/pipeline.py
92.13% <0.00%> (+0.06%)
:arrow_up:

reframe/frontend/loader.py
92.56% <0.00%> (+1.65%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 70daadc...dba32c1. Read the comment docs.

Also I would expect the actions using the different modules systems to fail 🤔


misc: ignore .idea files

Usually, we don't need to commit any file under .idea, some files changed on my local once and once again, be easy to mis-commit them.
Refs: https://github.com/spotbugs/spotbugs-gradle-plugin/pull/975#issuecomment-1753974908.

CC @hazendaz.

Thanks @Goooler !  I 100% agree on this one.  Anytime that stuff is checked in it makes rather rash assumptions that all users use same tool versions which is never accurate.  Never mind they are auto generated ;)


Use ray to parallelize read_feather with pyarrow feather API

What do these changes do?
Using ray to parallelize read_feather natively with pyarrow feather API
similar to what was done in read_parquet
in order to get the parallelization (across columns) I used the pyarrow API instead of pandas, similar to what was done in read_parquet.
one API issue, pyarrow supports use_threads while pandas nthreads.
to resolve this I did not remove nthreads from the API, but not using it, and using the default for use_threads in pyarrow which is currentlly True
Related issue number
#278

[x] passes git diff upstream/master -u -- "*.py" | flake8 --diff
[x] passes black --check modin/

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Modin-PRB/166/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Modin-Performance-PRB/104/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Modin-PRB/167/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Modin-Performance-PRB/105/
Test PASSed.

Thanks @eavidan! I am going to do some performance benchmarking on this branch to see what our speedup is. I will report back with some numbers.

After doing a bit of testing, this PR is significantly faster than pandas, even on reading the feather-format with nthreads set.
1GB data - np.random.randint(0, 100, size=(2**20, 2**8))
pandas.read_feather(fname, nthreads=144) - 3.57 sec
modin.pandas.read_feather(fname, nthreads=144) - 1.1 sec
4GB data - np.random.randint(0, 100, size=(2**22, 2**8))
pandas.read_feather(fname, nthreads=144) - 13.6 sec
modin.pandas.read_feather(fname, nthreads=144) - 2.05 sec
16GB data - np.random.randint(0, 100, size=(2**24, 2**8))
pandas.read_feather(fname, nthreads=144) - 56.5 sec
modin.pandas.read_feather(fname, nthreads=144) - 10.8 sec
Very nice, thanks!

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Modin-PRB/168/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Modin-Performance-PRB/106/
Test PASSed.


S3 Data Store raising Excon::Errors::Conflict while attempting destroy_content

When trying to save a new file as an attachment, I see save calls destroy_previous which results in an exception:
Excon::Errors::Conflict: Expected(204) <=> Actual(409 Conflict)
The error message is "The bucket you tried to delete is not empty".
I see this type of exception is not rescued on s3data_store.rb.
I'm not sure why it tries to delete the bucket instead of just a file.

Hi - thanks for this.
I'd rather raise a separate error than "DataNotFound", because the conflict error doesn't necessarily mean DataNotFound, so I've added a commit that raises a different error, which will be also rescued and logged by the model attachment - see https://github.com/markevans/dragonfly/commit/d8cc1a47a82beb591f59ba25ae37bfdf544381f2

Sounds perfect, thanks a lot for the fix :)


Cannot build with no-sandbox

I've spent many days trying to get forge to create a working Snapcraft build, and cannot seem to get it to work. It seems that if we could pass --no-sandbox to the Electron build, that it might help, but there doesn't seem to be a way to do it with forge, is there?

+1

@benjamingahery I have a PR open with a fix for this, but it hasn't received any feedback yet https://github.com/electron-userland/electron-installer-snap/pull/75

Thanks for the information! Hope it will come soon!

This will be fixed in electron-installer-snap 😄

how to run electron in dev mode with no sandbox ?


Remove errant mdash in C++/WinRT intro

Instead of appearing as a dash, it appeared literally as "&mdash", but since it's at the end of a sentence, it should be removed.

Yes, thanks, @plieblang. FYI, a semi-colon at the end would have made that valid markdown (—). I forgot to tidy that up when I refactored the sentence. Thanks for fixing it! :) ~Steve


add rescript support to vim-polyglot

https://github.com/sheerun/vim-polyglot is a popular repo for getting all languages support (so that you don't install each lsp individually)
should this be added to the list to make it simpler for newcomers to have rescript syntax support out of the box, without even installing any new plugins
for example reason config:
https://github.com/sheerun/vim-polyglot/blob/ce31cd1d2f4e8eee9fd91325e4599f15cb9566fd/packages.yaml#L1390-L1394
and ocaml:
https://github.com/sheerun/vim-polyglot/blob/ce31cd1d2f4e8eee9fd91325e4599f15cb9566fd/packages.yaml#L1105-L1117

this repo does more than just syntax highlighting (intellisense stuff and others) if it's okay for them to increase the installation size you can send a pr for them.


Rd/bfp8b pack

Ticket
Link to Github Issue
Problem description
bfp8 packing is inaccurate when pack_src_format is also bpf8, since this results in double rounding in the HW. First the gasket rounds to 7 bits, then rounding occurs again when the mantissas are being shifted in order to have common exponent.
What's changed
Add a flag to compute config called bfp_pack_precise which toggles the pack_src_format to either fp16 or fp32 (depending on fp32_mode_en) in order to get more accurate output. This however will half the packer bandwidth in the case of fp16, and reduce it to one quarter in the case of fp32.
Checklist

[ ] Post commit CI passes: https://github.com/tenstorrent/tt-metal/actions/runs/11805678987
[ ] https://github.com/tenstorrent/tt-metal/actions/runs/11805683575
[ ] https://github.com/tenstorrent/tt-metal/actions/runs/11805684681
[ ] https://github.com/tenstorrent/tt-metal/actions/runs/11805685869
[ ] https://github.com/tenstorrent/tt-metal/actions/runs/11805686731
[ ] https://github.com/tenstorrent/tt-metal/actions/runs/11805687445

yes, good point I will add those before merging

@rdjogoTT , please update PR title

@eyonland can someone from MCW help to bring this to binary too?


Support more languages!

Description of the new feature / enhancement
We hope to add more languages! Because China has a very large user base, but most people's English is not up to a fluent level
Scenario when this would be used?
As I said, when users from non-English popular countries use it.
Supporting information
No response

We support many langauges https://learn.microsoft.com/en-us/windows/powertoys/#languages
For Chinese, we support simplified and traditional.


Integration with MISP

Much like the crits integration, an integration module for MISP would be excellent. I may look into this if I have time.

Started looking into this already: managed to get a working version which captures the Event ID for the event which the sample is contained within. Happy to share my code if you would like.
I'm not the strongest when it comes to PHP, and have a little issue which would require more work, but the fundamentals seem to be working nicely.

For vt_runMISP.php
`<?
require('config.php');
$counter = 0;
$m = new MongoClient("mongodb://".$mongo_server_host.":".$mongo_server_port);
$db = $m->selectDB($mongo_db);
$collection = new MongoCollection($db, $mongo_collection);
$cursor = $collection->find();
if($misp_on == "true")
{
foreach ($cursor as $array)
{
$data = array('request' => array('value' => $array['md5'],"type" => "md5"));
$data_json = json_encode($data);
$context = stream_context_create(array(
'http' => array(
'method' => 'POST',
'header' => "Authorization:" . $misp_key ."\r\n" .
"Accept: application/json\r\n" .
"content-type: application/json\r\n",
'content' => $data_json
)
));
$url = $misp_url . "/events/restSearch/download";
$result2 = file_get_contents($url, false, $context);
$thejson = json_decode($result2, true);
if (is_array($thejson) && array_key_exists('response',$thejson)){
$jsoncount = count($thejson['response']);
if($jsoncount > 0)
{
$jsonevents = "";
$jsoncounter = 0;
foreach ($thejson['response'] as $jsonevent)
{
$jsoncounter++;
if($jsoncount==0 || $jsoncount == $jsoncounter)
{
$jsonevents .= $jsonevent['Event']['id'];
}
else
{
$jsonevents .= $jsonevent['Event']['id'] .', ';
}
}
$retval = $collection->findAndModify(
array("id" => $array['id']),
array('$set' => array('misp' => "true",'misp_event' => $jsonevents))
);
$counter++;
}
}
}
echo "Found " . $counter . " Samples in MISP";
}
else
{
echo "MISP config is set to False";
}
?>`

Code for vt_runMISP.php
<?
require('config.php');

$counter = 0;
$m = new MongoClient("mongodb://".$mongo_server_host.":".$mongo_server_port);
$db = $m->selectDB($mongo_db);
$collection = new MongoCollection($db, $mongo_collection);
$cursor = $collection->find();

if($misp_on == "true")
{
    foreach ($cursor as $array)
    {
        $data = array('request' => array('value' => $array['md5'],"type" => "md5"));
        $data_json = json_encode($data);
        $context = stream_context_create(array(
            'http' => array(
                'method' => 'POST',
                'header' => "Authorization:" . $misp_key ."\r\n" .
                            "Accept: application/json\r\n" .
                            "content-type: application/json\r\n",
                'content' => $data_json
            )
        ));
        $url = $misp_url . "/events/restSearch/download";
        $result2 = file_get_contents($url, false, $context);
        $thejson = json_decode($result2, true);
        if (is_array($thejson) && array_key_exists('response',$thejson)){
            $jsoncount = count($thejson['response']);
            if($jsoncount > 0)
            {
                $jsonevents = "";
                $jsoncounter = 0;
                foreach ($thejson['response'] as $jsonevent)
                {
                    $jsoncounter++;
                    if($jsoncount==0 || $jsoncount == $jsoncounter)
                    {
                        $jsonevents .= $jsonevent['Event']['id'];
                    }
                    else
                    {
                        $jsonevents .= $jsonevent['Event']['id'] .', ';
                    }
                }
                $retval = $collection->findAndModify(
                     array("id" => $array['id']),
                     array('$set' => array('misp' => "true",'misp_event' => $jsonevents))
                );
                $counter++;
            }
        }
    }
    echo "Found " . $counter . " Samples in MISP";
}
else
{
    echo "MISP config is set to False";
}
?>

Couple other small bits in config etc, but this is the bulk.
Pulls back correctly, but seem to get a lot of 404 failure requests for hits which aren't in MISP. Should come back with a response of "no results"

thanks, might need to hold off till I have a MISP to test against

Excellent, hope it goes successfully when you do get one set up.
Feel free to take a look at my active development of MISP integration.
https://github.com/tomking2/VT-Hunter/tree/MISP

Thanks for integrating, it's looking good. This can be closed now.


Refer to step check instead of non-existing step verify

What this PR does / why we need it:
Refer to step check instead of non-existing step verify
Special notes for your reviewer:
Follow-up/fix of PR  #109
Release note:

NONE

cc @ScheererJ


Make global attributes be able to be per-cluster

Add a global attribute per-cluster choice.
Currently we only have per-app or global choice.

Is this still an issue?


Update app-service-web-vnet-troubleshooting.md

Change delegation from Microsoft.Web to Microsoft.Web/serverFarms

@subashsivaji : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off


Wrong plural usage for time in PT-BR

There is a problem with the following definitions for Brazilian Portuguese:
    longDateFormat : {
        LT : 'HH:mm',
        LTS : 'HH:mm:ss',
        L : 'DD/MM/YYYY',
        LL : 'D [de] MMMM [de] YYYY',
        LLL : 'D [de] MMMM [de] YYYY [às] HH:mm',
        LLLL : 'dddd, D [de] MMMM [de] YYYY [às] HH:mm'
    },
    calendar : {
        sameDay: '[Hoje às] LT',
        nextDay: '[Amanhã às] LT',
        nextWeek: 'dddd [às] LT',
        lastDay: '[Ontem às] LT',
        lastWeek: function () {
            return (this.day() === 0 || this.day() === 6) ?
                '[Último] dddd [às] LT' : // Saturday + Sunday
                '[Última] dddd [às] LT'; // Monday - Friday
        },
        sameElse: 'L'
    }

The string "às" should only be used if the hour (in 24-hour format) is at least 2. If H == 0 or H == 1, "à" should be used instead.
I managed to find a solution for the calendar part (I only tested this by extending the locale with updateLocale):
    calendar:  {
        sameDay: function() { return '[Hoje] ' + (this.hours() < 2 ? '[à]' : '[às]') + ' LT'; },
        nextDay: function() { return '[Amanhã] ' + (this.hours() < 2 ? '[à]' : '[às]') + ' LT'; },
        nextWeek: function() { return 'dddd ' + (this.hours() < 2 ? '[à]' : '[às]') + ' LT'; },
        lastDay: function() { return '[Ontem] ' + (this.hours() < 2 ? '[à]' : '[às]') + ' LT'; },
        lastWeek: function () {
            return (this.day() === 0 || this.day() === 6) ?
                '[Último] dddd ' + (this.hours() < 2 ? '[à]' : '[às]') + ' LT': // Saturday + Sunday
                '[Última] dddd ' + (this.hours() < 2 ? '[à]' : '[às]') + ' LT'; // Monday - Friday
        },
        sameElse: 'L'
    }

However, I cannot do the same with longDateFormat, because its values apparently cannot be functions.
Is there a solution for this (by either editing pt_br.js or extending the locale externally)?
Thank you

I guess we'd need a code change to allow them to be functions

As a brazilian, I don't understand what is the problem here...
I think it's correct to say: "Hoje às 00:00" or "Hoje às 01:00/13:00", right?

Wrong.
In “à(s) xx:00” — which is read as “à(s) xx hora(s)” —, the “à(s)” is a contraction of the preposition “a” with the article “a(s)”. The article refers to “xx hora(s)”, so it cannot be plural if xx is less than 2. Therefore, we must say “à 0 hora”, “à 1 hora”, “às 2 horas”, “às 3 horas”, and so on.
For the same reason, if we want to say the current time (“it’s...”), we must use “é” (and not “são”) before  “zero hora”, “uma hora”, “meio-dia” and “meia-noite”.
If you have questions about this topic, I recommend checking some grammar, such as Moderna Gramática Portuguesa by Evanildo Bechara.
I’m Brazilian too, and I know that even in Brazil it’s common to hear these expressions used incorrectly. However, I think Moment.js should still display them properly.


Navigating the "Nouveaux tests positifs" figure

In the "Nouveaux tests positifs" figure, it would be helpful if the hover text contained the department ID. Adding the first few letters of the department name to the legend entries would also help.

Done!
Thank you for your message.


[improve][admin] Check if the topic existed before the permission operations

Motivation
The get/grant/revoke permissions to a non-existed topic will be ok which may confuse the user. So it's better to check the topic existed before the grant permissions.
Documentation

[ ] doc
[ ] doc-required
[x] doc-not-needed
[ ] doc-complete

I think this should be a breaking change and we need a PIP for it. Users need to explicitly create a topic before granting the permission after this change. This doesn't seem to make sense in some cases. The admin would simply want to grant users permissions without requiring that the topic has already been created.

not a break change. because new grant topic permissions need to create topic first. old ones not impact

@Technoboy- AuthorizationTest doesn't pass in branch-3.2 because this PR has been cherry-picked to branch-3.2 . This blocks 3.2.3 release. Would you be able to fix the problem? Thanks

@Technoboy- AuthorizationTest doesn't pass in branch-3.2 because this PR has been cherry-picked to branch-3.2 . This blocks 3.2.3 release. Would you be able to fix the problem? Thanks

Fixed. forgot to cherry-pick https://github.com/apache/pulsar/pull/22550

I think this should be a breaking change and we need a PIP for it. Users need to explicitly create a topic before granting the permission after this change. This doesn't seem to make sense in some cases. The admin would simply want to grant users permissions without requiring that the topic has already been created.

I agree, this is a breaking change. There are reports from Pulsar users that this breaks their Pulsar solutions.
@Technoboy- We  need to restore the old behavior and possibly have a toggle for this new behavior.

The breaking change introduced by this PR is being addressed in #23709.


Skill - Armsthrift

Luck % chance to not consume weapon durability for this attack.

This modification does not take into account either indestructible weapons or increased attrition due to combat skills, but as this feature is more interesting,I may fix it after merging


[v1.12] Backport print-chart-version script for dev Helm chart push

[ upstream commit d90803cec8e7a1508558b183600cde7dcbdd719f ]
[ upstream commit 2c367de92d1825bf2ab80de2fd4e848ee2e28c46 ]
[ Backporter's notes:

Removed github workflow. Workflow operates from main branch.
Applied /usr/bin/env hunk to the script (was treewide). ]

Related: #25205
$ for pr in 25205; do contrib/backporting/set-labels.py $pr done v1.12; done

✅ https://github.com/cilium/cilium/actions/runs/5224591407/jobs/9432967787


Disable log4j message lookup

The docker solr images were updated to include the argument -Dlog4j2.formatMsgNoLookups=true to mitigate the log4j vulnerability, see their github page 1 and 2
The argument is only set when you start your solr container with "solr-fg" or "solr-foreground" or "start-local-solr"

I am not super familiar with how Docker works. What is the difference between exec solr -f and exec solr -fg?
Incidentally, CVE-2021-45046 suggests that the fix implemented for the Docker images may not totally mitigate the vulnerability.

About if the image itself was patched:
I'm going to summarize what I read following solr github repo, docker-solr github repo and solr message board:

The docker solr images were updated to include the argument -Dlog4j2.formatMsgNoLookups=true to mitigate the log4j vulnerability, but only in their included startup scripts solr-fg and start-local-solr , see commit history of docker-solr
When the following vulnerability was disclosed (the one Benjamin mentioned), solr stated that they are not affected, and a pull request that removed the JNDILookup.class was rejected by the docker-solr maintainers since setting the  -Dlog4j2.formatMsgNoLookups=true argument is deemed sufficient.
There were also no commits to the Dockerfile of docker-solr 7.7 since Jun2020 and one of the maintainers commented here:

Without that, all you need to do is pass a system property. You certainly needn't go editing files in the image to do this! Set SOLR_OPTS at your service definition

I also tested using the docker image (after pulling the new docker-solr image) and when I go to "Java Properties" on the sidebar, the argument is not included, only if I change the line to exec solr-fg  or  exec solr -f -Dlog4j2.formatMsgNoLookups=true

Alternative:
Again if you have any doubts about using the startup scripts included in docker-solr, I can change the commit to only include the argument with the command like exec solr -f -Dlog4j2.formatMsgNoLookups=true  which would also be sufficient

The version of solr 8.11.1 is now available with that change . It replaces 8.11.0 which is no longer online
I have personally tried solr 8.11.1 on vivo 1.12.1 and it seems to work fine. I will get back to you if I detect a problem

I'm aware that solr 8.11.1 is out (see VIVO slack), and this is a temporary fix. Updating the Dockerfile is an option but will need more work than changing the version number, see the README of docker-solr:

Updating from Docker-solr5-7 to 8
For Solr 8, the docker-solr distribution switched from just extracting the Solr tar, to using the service installation script. This was done for various reasons: to bring it in line with the recommendations by the Solr Ref Guide, to make it easier to mount volumes, and because we were asked to.
This is a backwards incompatible change, and means that if you're upgrading from an older version, you will most likely need to make some changes. If you don't want to upgrade at this time, specify solr:7 as your container image. If you use solr:8 you will use the new style. If you use just solr then you risk being tripped up by backwards incompatible changes; always specify at least a major version.
Changes:
The Solr data is now stored in /var/solr/data rather than /opt/solr/server/solr. The /opt/solr/server/solr/mycores no longer exists
The custom SOLR_HOME can no longer be used, because various scripts depend on the new locations. Consequently, INIT_SOLR_HOME is also no longer supported


fix: get rejectors from queued transactions

Resolves #456
Rejectors were previously only fetched from executed transactions but they are now retrieved solely based on nonce, therefore including queued transactions.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.07%) to 92.148%

Totals

Change from base Build<PHONE_NUMBER>:
0.07%

Covered Lines:
4137

Relevant Lines:
4381

💛  - Coveralls


Issues #123 - #124

Hi, regarding to #123 and #124,

I updated themes file to allow gradient and image background values.
I updated layouts file to be compatible with the first one, because previously there are no # character in color values in themes file so it was being added automaticly in the layouts file assuming every color value is hex.
I updated structures of churchill and samuel layouts because they were not compatible with gradient and/or image backgrounds.

Here are examples:

@nishantpersonal : Kindly review the PR and check it
@dolmushcu : Will update you since there are few major changes, we will test and ensure existing functionalities are not affected


Bugs/issues in ODE analysis functions

Non-exhaustive list of bugs/issues I have found with the ODE analysis and related functions which need fixing:

get_right_hand_side() passes covariates into predict as a dataframe instead of an interpolation as expected
vectorfield2d and nullclines2d do not work with MultiUDEs
vectorfield2d and nullclines2d are hard-coded to only work for models with 2 state variables (intentional?)
vectorfield_and_nullclines only works for models with 2 state variables due to above (if intentional should change name to vectorfield_and_nullclines2d)
the values that lower and upper represent needs to be clarified in function descriptions alongside clarification about their formatting (I assume an array with one element per state variable?)

Closing as I am not finding issues referred to above and think I might have been looking at an outdated version of the package. I am going to run through and re-test vectorfield2d and nullclines2d and reopen independent issues as needed


Connect a Controller via Bluetooth

Get a controller, PS4, PS3, etc. working with the ROS Node over bluetooth.

Once this is figured out, please freeze the version of ds4drv in the setup.py


Update Set-SPManagedAccount.md

Format example

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

sharepoint/sharepoint-ps/sharepoint-server/Set-SPManagedAccount.md
:white_check_mark:Succeeded
View (sharepoint-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Doesn't work with strings

In order for it to work with strings one must manually convert strings to buffers before passing them to this module.

I'd happly merge a pull request that added support for other encodings, in a way that is compatible with node's fs write steam.

Sure. The README says it accepts strings though, so I assumed this was already supported 😁

oh i see it, but it also says "currently really simple and does not yet support all fs.createWriteStream options yet."


WinUI: Support for unpackaged apps

Unpackaged apps don't use MSIX. They're typically installed and updated using .exe or .msi files. Many unpackaged apps walk end-users through an installation wizard using a classic UI.
https://github.com/dotnet/maui/issues/3166

What's the advantage of this? This just seems like a downgrade in terms of ease of deployment and automatic updates.

Hi @charlesroddie
Unpackaged apps are completely unrestricted in terms of the APIs they call, access to the registry, and file system. They aren't for everyone.
At the same time, packaged apps do slow down the inner-dev loop. So it should be possible to set an app as unpackaged while it's being developed (for faster builds), then switch it to packaged towards the end of the development process.

We've had a doc on publishing unpackaged apps for a while now.


Fix typo

Ripgrep is great too though.

Oops! No prizes for guessing which other projects I looked at for examples of install instructions :).
Fixed in #665.


version update

Update terraform required min version.

It's great having you contribute to this project Welcome to the community :nerd_face: :rocket:Soon one of our maintainers will review it and provide you with feedback/suggestions. If you think it's something urgent, feel free to reach out Md Shamim on LinkedIn. Please star ⭐ this repo to show us support.Happy, Open Source!

This issue has been resolved in version 1.1.3 :tada:


Double Parenthesis in Query Interpolation with List Parameters for IN clause

Summary:
When using the Databricks SQL Python integration to interpolate list parameters into a SQL query, the list parameters are incorrectly wrapped in double parentheses. This results in a malformed SQL query, causing execution errors on the Databricks side.
Details:
I encountered an issue while using the Databricks SQL Python integration to dynamically inject a list of values into a SQL query. The expected behavior is to interpolate the list parameters into the query, properly formatted for an IN clause. However, the interpolation wraps the list in an extra set of parentheses, leading to a Databricks runtime error.
Query template
query = """
select *
from SOME_TABLE
where ID IN (%(id_list)s)
"""
parameters = {'id_list': [1, 2, 3]}
# Interpolation process here

Expected:
select *
from SOME_TABLE
where ID IN (1, 2, 3)

Actual:
select *
from SOME_TABLE
where ID IN ((1, 2, 3))

I know that removing outer parathesis will solve the error, but the same query is used by Snowflake connector which requires lists to be wrapped in parenthesis.
By the way, it might be the other way around and I might need to submit bug to Snowflake community.
Let me know what you think.

Thanks for opening this issue. I've started a discussion internally to see if we should / how we can address this.

By the way, it might be the other way around and I might need to submit bug to Snowflake community.

I don't think this is really a "bug" in either connector. It's just a difference in syntax. We mention in the docs for inline sequence parameters that the precise behaviour of this syntax is beyond the scope of the PEP-249 standard. Which means connector developers are free to implement however they deem fit.

Thank you!
Will be monitoring this thread for updates.

Thanks. We have a few folks out for end-of-year so it will be a few days before we can get the right eyes on this.

Good morning @susodapop
Do you know if there are any updates here?
Thank you!


feat(buildkit): Support build-time HTTP_PROXY

Description
			FrontendAttrs: map[string]string{
				"build-arg:HTTPS_PROXY": os.Getenv("HTTPS_PROXY"),
			},

Buildkit supports some build time attrs in its dockerfile frontend. We need to support them in envd.

I will work on this.

Ref https://github.com/moby/buildkit/issues/971