Schema options in params(...) are ignored

Hi utoipa folks! First of all, thank you very much for your great work on this library, it has helped us a bunch to manage our API spec!
We're just trying to update from v3.3.0 to v4.1.0, and it seems that additional schema options in params(...) inside of the #[utoipa::path(...)] now get ignored 🥲
We're using axum and have the axum feature in utoipa turned on.
For example, we have an endpoint similar to this (reduced to essential stuff):
#[utoipa::path(
    get,
    path = "/user/{user_id}",
    params(
        ("user_id", min_length = 1),
    )
)]
#[tracing::instrument(err(Debug))]
async fn get_user(
    AxumState(state): AxumState<State>,
    Path(user_id): Path<String>,
) -> Result<Json<User>> {
  // . . .

With v3.3.0, this used to emit something like this in the OAPI doc:
  /user/{user_id}:
    get:
      operationId: get_user
      parameters:
      - name: user_id
        in: path
        required: true
        schema:
          type: string
          minLength: 1

With v4.1.0, the minLength: 1 gets removed from the OAPI doc output :(
Randomly, I also tried adding the nullable annotation to the same parameter, i.e. turning it into:
    params(
        ("user_id", min_length = 1, nullable),
    )

In v4.1.0, this didn't affect the output at all.
In v3.3.0, this changed the output as expected:
  /user/{user_id}:
    get:
      operationId: get_user
      parameters:
      - name: user_id
        in: path
        required: true
        schema:
          type: string
          nullable: true
          minLength: 1

Is this a known issue? I looked around in the issues, but couldn't find one that exactly seems to match this problem (but could be that the description/title is too different.)
The only slightly similar issue I could find is https://github.com/juhaku/utoipa/issues/693, but I'm not sure if that one might be a separate problem affecting only format 🤔
If there's something we can do to help isolate this, please let me know!

Thanks for reporting this. The issue you referred might be something similar. E.g. In one the parameters behave differently when the actix_web or other integration feature flag is enabled. This most likely needs some investigation whether the case is same here as well. There are quite a few variations when it comes to the attribute parsing and schema generation so it might be that there is some edgecase that does not handle the parsed attributes correctly. E.g. by "removing / nulling" the attributes even when it should not do that.

Thanks for your quick response, and sorry for reacting so late myself.

E.g. by "removing / nulling" the attributes even when it should not do that.
Yes, that's what it feels like 🤔

Since this issue removes attributes from our API spec that we need to keep, this currently blocks us from upgrading to utoipa 4.
Is there some way I can help with this?
I'm not familiar with utoipa's codebase, and I also don't have experience with writing (and debugging) macros and code generation in Rust 😅
So I don't think it's realistic I can try to isolate and fix this bug directly. 🙇

Hey again @juhaku,
I did some investigation around the code, and I think I've found one way to fix a simple test case (like the one I mentioned above, a string parameter with a min_length feature), but I'm not sure how robust it is, so you should definitely also look at this yourself (and we should maybe add more test cases).
The simple test case can look like this:
// in utoipa-gen/tests/path_derive_axum_test.rs

#[test]
fn path_param_primitive_arg_with_additional_settings() {
    #[utoipa::path(
        get,
        path = "/items/{id}",
        params(
            ("id", min_length = 1),
        ),
        responses(
            (status = 200, description = "success response")
        )
    )]
    #[allow(unused)]
    async fn get_item(id: Path<String>) {}

    #[derive(OpenApi)]
    #[openapi(paths(get_item))]
    struct ApiDoc;

    let doc = serde_json::to_value(ApiDoc::openapi()).unwrap();
    let operation = doc.pointer("/paths/~1items~1{id}/get").unwrap();

    println!("{:?}", operation.pointer("/parameters"));
    assert_json_eq!(
        operation.pointer("/parameters"),
        json!([
            {
                "in": "path",
                "name": "id",
                "required": true,
                "schema": {
                    "type": "string",
                    "minLength": 1, // without the fix, this property will be missing
                }
            }
        ])
    )
}

As you suggested, the main issue seems to be that some information from the explicit parameter info is overwritten with information extracted from axum:
the merge function in Parameter doesn't update the features fields of parameter_schema and self.
Basically, it needs to repeat these three lines from the Parse function:
https://github.com/juhaku/utoipa/blob/1ea9cf84b23a431939c34d918fb07b1c6965cf71/utoipa-gen/src/path/parameter.rs#L262-L265
I'm not sure if we also need to move the features from other.features to self.features first.
The above test case works with this adjusted version of the merge function:
    pub fn merge(&mut self, other: Parameter<'p>) {
        match (self, other) {
            (Self::Value(value), Parameter::Value(other)) => {
                value.parameter_schema = other.parameter_schema;

                // not sure if this is needed:
                let (mut schema_features, mut param_features) = other.features;
                value.features.0.append(&mut schema_features);
                value.features.1.append(&mut param_features);

                // same logic as in Parse():
                if let Some(pf) = &mut value.parameter_schema {
                    pf.features = value.features.0.clone();
                }
            }
            (Self::IntoParamsIdent(into_params), Parameter::IntoParamsIdent(other)) => {
                *into_params = other;
            }
            _ => (),
        }
    }

I'd also like some feedback from exploring and fiddling with the code (but I'm aware that I just dove in for some very isolated things, so feel free to ignore this if you think otherwise):
The modeling of the "features" properties was quite confusing for me, and seems to be somewhat fragile.
As a pure refactoring, I recommend making  ValueParameter.features easier to understand by splitting the tuple, so each of the two vectors can have its own name. For example, turning it into two fields (schema_features: Vec<Feature>, param_features: Vec<Feature>) could work. Or if the two should remain connected (e.g. sometimes used in an Option), a sub-struct with named parameters would help.
Furthermore, it seems to be errorprone that the schema_features need to be copied into the parameter_schema (but only if parameter_schema is not None?). If possible, it would be good to keep it only in one place.
However, I realize that this might be difficult to do, depending on how the code works around it.
Finally, I wonder if it would make the code easier to understand to split Feature into separate types, one for parameter features and one for schema features. However, I also realize this might make the code more annoying in some places; I especially don't know how this interacts with parsing the macro input.

Hi @juhaku! Happy new year! 🎉
Just wanted to check in, are there any updates on this?
Unfortunately this is blocking us from upgrading to utoipa v4 :(

Hi @juhaku! Happy new year! 🎉
Just wanted to check in, are there any updates on this? Unfortunately this is blocking us from upgrading to utoipa v4 :(

Hello, and sorry for this taking so long, :disappointed: I am now taking this under work.

The above test case works with this adjusted version of the merge function:
   pub fn merge(&mut self, other: Parameter<'p>) {
       match (self, other) {
           (Self::Value(value), Parameter::Value(other)) => {
               value.parameter_schema = other.parameter_schema;

               // not sure if this is needed:
               let (mut schema_features, mut param_features) = other.features;
               value.features.0.append(&mut schema_features);
               value.features.1.append(&mut param_features);

               // same logic as in Parse():
               if let Some(pf) = &mut value.parameter_schema {
                   pf.features = value.features.0.clone();
               }
           }
           (Self::IntoParamsIdent(into_params), Parameter::IntoParamsIdent(other)) => {
               *into_params = other;
           }
           _ => (),
       }
   }

True, that indeed fixes this issue with params, thanks for noticing. That above both is needed since that will be needed in ToTokens implementation ValueParameter to check whether user has defined features for parameter type without defining parameter type itself.


Invalid parameters.

Whenever I'm trying to extract the video, I get an error saying "Invalid Parameters".
My Code:
private Context context;
    private YouTubeExtractor mExtractor;
    private Callback<YouTubeExtractionResult> mExtractionCallback;
    private String id;

    public Downloader(Context context, String id) {
        this.context = context;
        this.id = id;

        mExtractor = YouTubeExtractor.create();

        mExtractionCallback = new Callback<YouTubeExtractionResult>() {

            @Override
            public void onResponse(Call<YouTubeExtractionResult> call, Response<YouTubeExtractionResult> response) {
                bindVideoResult(response.body());
            }

            @Override
            public void onFailure(Call<YouTubeExtractionResult> call, Throwable t) {
                onError(t);
            }
        };
    }

    public void download() {
        mExtractor.extract("9d8wWcJLnFI").enqueue(mExtractionCallback);
        mExtractor.extract(this.id).enqueue(mExtractionCallback);
    }

    private void onError(Throwable t) {
        t.printStackTrace();
        Log.d("OnError", "id: " + id);
        Toast.makeText(context, "It failed to extract. So sad", Toast.LENGTH_SHORT).show();
    }

    private void bindVideoResult(YouTubeExtractionResult result) {
        Log.d("OnSuccess", "Got a result with the best url: " + result.getBestAvailableQualityVideoUri());
        Toast.makeText(context, "result : " + result.getSd360VideoUri(), Toast.LENGTH_SHORT).show();
    }
}

Looks like you are using an older version of this library. This issue is more than likely fixed and you should test with the latest to see if it is still an issue for you.

What is the latest version?
By the way, I'm using 2.1.0 :)

4.2.1, instructions are in the README for how to use the latest.

Oh, is it only for Kothlin?

It is written in Kotlin, but you can use it in Java

YouTubeExtractor extractor = YouTubeExtractor.create();
YoutubeExtractor doesn't have a method called create();
YoutubeExtractor.Companion does have it though.

Yep, certain calls will be different from the example when using Java instead of Kotlin

After getting the result
(
        YouTubeExtractor extractor = YouTubeExtractor.Companion.create();
        extractor.extract("9d8wWcJLnFI").blockingGet();

)
How can I convert it into a downloadable?


IE 11 issues

IE11 is not compatible.
Launch IE, go to http://bvaughn.github.io/react-virtualized-select
I get the following error:
Minified React error #39; visit http://facebook.github.io/react/docs/error-decoder.html?invariant=39&args[]= This may be caused by unintentionally loading two independent copies of React. for the full message or use the non-minified dev environment for full errors and additional helpful warnings.
When I debug locally with non-minified React then I get this:
Objects are not valid as a React child (found: object with keys {$$typeof, type, key, ref, props, _owner}). If you meant to render a collection of children, use an array instead or wrap the object using createFragment(object) from the React add-ons. Check the render method of Select.
When I use "react-select" instead of "react-virtualized-select" then component is working fine.
Found similar issue reference: https://github.com/facebook/react/issues/8379
It seems to be related to babel polyfill and load order ?

Should be resolved now. Just needed to polyfill a few things (eg symbol, promise) for IE.


Added support for retina images using srcset in img tag.

Used by adding srcset string when pushing images to gallery.

Example:
slides.push({
	thumb: image.getThumbnailUrl(),
	image: image.getWebImageUrl(),
	big: image.getSlideUrl(),
	title: gallery.getName(),
	description: image.getDescription(),
	srcset: image.calculateSrcSet()
});

Results in:
<img width="921" height="691" style="image-rendering: optimizequality; display: block; opacity: 1; min-width: 0px; min-height: 0px; max-width: none; max-height: none; transform: translate3d(0px, 0px, 0px); width: 921px; height: 691px; position: absolute; top: 0px; left: 490px; cursor: pointer;" src="slideshow/2014-10_062.jpg" srcset="thumbnails/thumb_2014-10_062.jpg 200w, pics/2014-10_062.jpg 800w, slideshow/2014-10_062.jpg 1600w, 2048x2048/2014-10_062.jpg 2048w, 3840x2160/2014-10_062.jpg 2880w" sizes="921px">

This will tell the browser that the image will be rendered at 921px width (initially), calculated by the Galleria scaler, and that there is a set of sources available. On my Nexus 9, it will choose the 2048w image.

After closer inspections of the network traffic, it seems the preloading is still using the smaller image. This causes every image to be loaded twice. One for retina, and one normal. I haven't been able to figure out where to connect this to the preloading functionality.

Does anyone currently have plans to continue working on this?

Anybody? No?


twig_template_get_attributes tests and fixes

This enables testing of twig_template_get_attributes (every TemplateTest test is ran against twig_template_get_attributes too) and fixes some bugs I've found.
There is still 3 failing tests due to a difference of how twig_template_get_attributes and Twig_Template::getAttribute use Twig_Template::$cache.
Twig_Template::getAttribute doesn't cache properties, and twig_template_get_attributes expects $cache[$class]['properties'] to be set if $cache[$class] is set. So twig_template_get_attributes can fail if the cache is already populated by Twig_Template::getAttribute.

Thanks.

The cache for the properties has been removed in the PHP version as it was not needed.

I've fixed the unit tests by clearing the cache each time we run a new test: 317f6e47256f3. I think it is a good idea anyway to reset the cache for each test.


Data selection and styling issues in Global Output's Overview

I'm not sure if 3dcae686c7877b8a40025589b59abeb02387fc19 fixes part of this

@mrkwanzaa, it does! Thank you.


Failed to delete tag

RT.

Version:
answer version 1.0.7
revision: 0c817fd4
build time: 2023-03-23T10:57:08Z

Please provide a test link so that we can locate the problem.

That's my local environment. At that time, I saw that Chrome did not send a request, which should be a front-end issue, but unfortunately, it could not reproduce stably.


How can I get access to the system's current active window and application?

Hello!
Is there a way currently to be able to monitor the current active application in the OS?
For example if I'm focused on chrome I want my tauri app to be able to pick that up.
Electron has an API that's able to give that information for programs that have a GUI window, called: desktopCapturer

There isn't a way to do that yet. Maybe you can find a Rust crate that exposes that?

For anyone wondering - I've been using https://github.com/dimusic/active-win-pos-rs to get information of the active window


Use NewYAMLReader instead of NewDocumentDecoder

Confusingly, NewYAMLReader also supports reading multiple documents, but internally it does not use an instance of bufio.Scanner. This effectively works around the issue hit previously where we exceeded the max token size for bufio.Scanner
This fixes the conformance test failures currently seen in master.

🤔 🐛 You appear to be fixing a bug in Go code, yet your PR doesn't include updates to any test files. Did you forget to add a test?
Courtesy of your friendly test nag.

This change fixes a test failure in post-submit (specifically conformance tests for galley).

@selmanj: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

integ-mixer-k8s-tests_istio
167f9a38d7700ec3e1b166fec9d70a5ab83acac1
link
/test integ-mixer-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

@selmanj: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

integ-mixer-k8s-tests_istio
167f9a38d7700ec3e1b166fec9d70a5ab83acac1
link
/test integ-mixer-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Implement patch() for calendar settings.

Fixes https://github.com/jay0lee/GAM/issues/42 by introducing command:
gam calendar <calendar email> metadata description|location|summary|timeZone <string>

Here is a more general approach:
def doCalendarModify():
  calendarId, cal = buildCalendarDataGAPIObject(sys.argv[2])
  if not cal:
    return
  body = {}
  i = 4
  while i < len(sys.argv):
    myarg = sys.argv[i].lower().replace(u'_', u'')
    if myarg == u'description':
      body[u'description'] = sys.argv[i+1]
      i += 2
    elif myarg == u'location':
      body[u'location'] = sys.argv[i+1]
      i += 2
    elif myarg == u'summary':
      body[u'summary'] = sys.argv[i+1]
      i += 2
    elif myarg == u'timezone':
      body[u'timeZone'] = sys.argv[i+1]
      i += 2
    else:
      systemErrorExit(2, '%s is not a valid argument for "gam calendar <email> modify"' % sys.argv[i])
  callGAPI(cal.calendars(), u'patch', calendarId=calendarId, body=body)

      elif argument == u'modify':
        doCalendarModify()

@jalmeroth can you try an approach like @taers232c suggests which allows multiple attributes to be set within 1 API call?

closing this as Ross already submitted his version


Doc improvement auditLoader doc

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buidlall

run buidlall

run buildall


feature/compare-button

Adding a compare button on top of the countries searcher

There seems to be a bug here. When I click on the compare button, and then I select a location on the first selector for locations, I get a javascript error that breaks the page. If after that I refresh the page it works. So there might be something not being initialized.

@pjosh , @simaob I've added some conditions on componentDidUpdate of NDCs content overview Provider to avoid the crash.
Everything is looking fine from my perspective.


Fix stacktrace for runtime errors

Purpose
Fixes https://github.com/ballerina-platform/ballerina-lang/issues/15538
Fixes https://github.com/ballerina-platform/ballerina-lang/issues/17072
Check List

[x] Read the Contributing Guide
[ ] Required Balo version update
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[x] Added necessary tests

[x] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Codecov Report

Merging #17643 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #17643   +/-   ##
=======================================
  Coverage   14.82%   14.82%
=======================================
  Files          46       46
  Lines        1214     1214
  Branches      193      193
=======================================
  Hits          180      180
  Misses       1020     1020
  Partials       14       14

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c138b54...8e86994. Read the comment docs.


Add Jayo Cardboard PLA Spool

Added Jayo Cardboard Spool

Hi, thank you for your submission, but could you please add the three measurements and if possible also a photo on which the spool is not cropped? Thank you.

added dimesions, but I don't have another photo because the spool is already in the trash and really dirty

Thanks, merging!


Add control of tab stop to code blocks via additional arguments to ```

Summary
Add support for additional parameters to code blocks delimieted by comma, space or such, so that tab stops can be controlled by adding ,ts=<n> after the language, where <n> is a number. for example: ```sh,ts=8
History and Rationale
Code blocks always presume that you are using a tab stop of four, though this is by no means universal. The bitter dispute over how many spaces a tabulation character should represent dates back to the late Cambrian where coccolithophores had originally defined as 8, but non-avian dinosaurs (m$) later decided it should be four.
Different projects use different coding standards and copying and pasting code into Zulip (or any Markdown) shouldn't require you to spend several minutes converting it all into spaces for it to appear as it does in ones IDE or text editor.
As it happens, GitHub has chosen 8 spaces, the same as the C coding standard for Linux (the kernel) as well as the POSIX definition.
struct jack {
	int		see_jack;
	char		see_jack_run;
	const char	*jack;
};

But the same code copied and pasted into Zulip looks like this
struct jack {
    int     see_jack;
    char        see_jack_run;
    const char  *jack;
};

Proposed Solution
The correct solution is to allow the tab stop to be specified in the same way as the language. Additionally, the language should also infer a default tab stop based upon the language. For example, C# is a Microcrap language and should default to the Microcrap standard of four, but still allow it to be overridden with c#,ts=3 (or even c# ts=3 )
using System.Hammer;

public class BeatMyFaceIn {
   private Hammer _withThis { get; }
   public void weRippedOffJava(Because thatsWhatWeDo) {
      return;
   }
}

Thanks for the detailed report! I agree it'd be nice to be able to control this in code blocks for those projects still using tabs. (It's certainly a declining market share, but we aim to make Zulip work for everyone!)
Are there other Markdown implementations that support this sort of syntax? We don't like making up our own syntax when there's prior art or a standard we can follow. https://meta.stackoverflow.com/questions/403016/literal-tab-in-code-block-renders-as-four-spaces suggests that Stack Overflow hasn't attempted to solve this problem, for example.

Hi,
I appreciate you raising the issue regarding Markdown rendering and tab customization. As a newcomer to the project, I'm excited to contribute and offer my thoughts on potential solutions.
•	After reviewing the discussion, I'd like to propose a potential solution. Given that Markdown rendering is handled by both Marked.js and Python-Markdown libraries, one approach could involve extending these libraries to support tab customization within code blocks. This would require modifications to recognize and interpret tab characters within fenced code blocks, enhancing the rendering functionality accordingly.
marked.js:
•	Modify marked.js to support the parsing of Markdown code blocks with customizable tab handling. This may involve adding functionality to parse additional parameters in code blocks, such as specifying the tab width (ts=).
Python-Markdown:
•	Extend or modify the Markdown parser in Python-Markdown to recognize and handle the custom tab handling syntax (ts=). This would involve updating the parsing logic to interpret the tab width parameter within code blocks.
By making changes to both libraries, we ensure consistency between the Markdown parsing on the frontend (using marked.js) and the backend (using Python-Markdown), allowing users to customize tab handling within code blocks across our entire project.
This can address the concerns mentioned previously by ensuring the :
Support for Projects Using Tabs:
•	Enabling users to control tab handling within code blocks accommodates projects that still use tabs for indentation, ensuring inclusivity and functionality for a wider range of projects.
Prior Art and Standards:
•	Modifying existing Markdown implementations like marked.js and Python-Markdown follows prior art and standards in the Markdown community, ensuring compatibility with established conventions and promoting interoperability with other Markdown-based platforms and tools.
In conclusion, modifying marked.js and Python-Markdown to support customizable tab handling addresses your concerns while promoting consistency, compatibility, and adherence to established standards within the Markdown ecosystem.
         •	Additionally, I propose opening separate issues or feature requests in the repositories of Marked.js and Python-Markdown. This would serve as a suggestion to the maintainers of these open-source libraries to consider implementing tab customization features. By engaging with the maintainers directly, we can explore opportunities to enhance the functionality of these libraries for the wider community


cosign: 0.3.1 -> 0.5.0

Motivation for this change
https://github.com/sigstore/cosign/releases/tag/v0.4.0
https://github.com/sigstore/cosign/releases/tag/v0.5.0
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[x] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Added a release notes entry if the change is major or breaking
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review run on x86_64-linux 1

1 package built:

cosign

We should probably backport this to stable, signatures made before the 0.4.0 are not compatible with the signature made after so it might be an annoyance in the future.

We should probably backport this to stable, signatures made before the 0.4.0 are not compatible with the signature made after so it might be an annoyance in the future.

that also means anyone using the tool right now couldn't use it after the update.

that also means anyone using the tool right now couldn't use it after the update.

Cosign is still in a heavy development mode so that should not be a surprise to anyone at this stage.
@06kellyjac WDYT?

Yeah I think anyone using cosign right now is or should be aware it's pre-release software and they're on the cutting edge. That's why the GH releases are marked pre-release and it's a 0.X.X semver which allows breaking changes.
If anyone is using it on stable then it'd be best if actually breaking changes are backported, but I'd also prefer if users followed unstable or master specifically for this package


Removed login button from other pages

As mentioned in the issue #12
Removed the login button after we login to the flavor feet ,
removed it from various html files.
Have a look

Merge it if it's ok @sagnik3788

give @ayan-joshi  25 points

give @ayan-joshi 25 points


#75: Straight-forward implementation of GCP Secret Manager support

For my project, I needed Secret Manager support for Quarkus native, so I decided to try to figure it out myself.
This is an implementation heavily inspired by the pubsub- and firestore-implementation. It's also an implementation that's very close to the original Google client library in user experience.
So I'm drafting this to hear your thoughts on it. Totally understand if it's too early.

I added a comment on the other PR to ping the contributor, if he didn't respond we should proceed with this one

I added a comment on the other PR to ping the contributor, if he didn't respond we should proceed with this one

So, #3 is abandonned and can be used as an inspiration.
I'm OK to proceed with this one, I'll review it as soon as it's ready.

So, #3 is abandonned and can be used as an inspiration.
I'm OK to proceed with this one, I'll review it as soon as it's ready.

@loicmathieu , @dzou : I think this is ready for review now. I've added a readme file. I've also looked through the #3 PR and think that I've addressed the relevant comments from that one as well.

@loicmathieu , @dzou : I think this is ready for review now. I've added a readme file. I've also looked through the #3 PR and think that I've addressed the relevant comments from that one as well.

Just saw that auth is enhanced on the PR from @dzou so this part is OK for this PR 👌

Just saw that auth is enhanced on the PR from @dzou so this part is OK for this PR 👌

Thanks
I add some documentation related comments.
Can you also add in the documentation the step to create the secret used in your example?
I wonder how it works regarding authentication? It didn't use the gcp credentials provided by the common lib.

Thanks for providing feedback! Just pushed an updated version of the readme file.
Also thanks to @dzou for improving the auth part, which I didn't put that much focus on. I've been using the GOOGLE_APPLICATION_CREDENTIALS environment variable locally, so that way it has been working on my machine.

Thanks
I add some documentation related comments.
Can you also add in the documentation the step to create the secret used in your example?
I wonder how it works regarding authentication? It didn't use the gcp credentials provided by the common lib.

Thanks for providing feedback! Just pushed an updated version of the readme file.
Also thanks to @dzou for improving the auth part, which I didn't put that much focus on. I've been using the GOOGLE_APPLICATION_CREDENTIALS environment variable locally, so that way it has been working on my machine.

@all-contributors please add @madsop for code and doc


Fix multi-move glitch

Commented out a line in recursive move_node call that causes a group of moved nodes to become nested within each other. This doesn't seem to be present in the the minified version.

Right below the line you commented out you will see the position being changed to after - what this means is that the first node is moved into place and all other nodes are moved after (not inside) the first moved node. I am afraid your fix will actually move all nodes after the first one outside the desired drop parent.
Could you please provide a demo of the issue and I will reopen? I have prepared a small jsfiddle so that we can discuss - the bottom two nodes are moved inside the first node. The demo uses the non-minified JS file of the latest version (3.3.4):
http://jsfiddle.net/DGAF4/2256/

Ah, I see now. We are using two separate instances of JQuery in our application, so the regex that determines the insert mode was broken when we replaced all of the "$". Thanks for your help!


mc-bootstrap adaptions for CAPV

this task is to adapt mc-bootstrap to create Management Clusters for Vsphere

This PR is enough to create a CAPV MC https://github.com/giantswarm/mc-bootstrap/pull/453

I merged the PR. We are able to create a CAPV MC now. If we need to implement other additional features, we can track separately.


Check OCaml-CI

Automated changes by create-pull-request GitHub action

Need to fix the dockerfile


Improve fatal error handler message for uncaught errors

The fatal error handler is intended as a "last resort" handler for assertion failures, uncaught errors, etc. It's a very bare bones mechanism and tries to minimize the risk of triggering further issues (such as segfaults) before the fatal error has been handled by user code.
In early development user code often makes Duktape API calls without a protected call wrapper. Uncaught errors in such situations will trigger the fatal error handler with a relatively cryptic message.
Improve this behavior so that uncaught errors are string formatted for the fatal error handler, so that it's easier to figure out the source of the error. The error formatting should not trigger further errors so it must be done carefully.
An alternative approach would be to provide a hook for uncaught errors which would be called before an uncaught error becomes a fatal error. The hook could be an Ecmascript or a Duktape/C function which could e.g. log the error to stderr before the fatal error handler gets called.

I was wondering about this, whether it was a bug or not.  In very early minisphere development I originally just set up a fatal error handler and then did an unprotected initial call into Duktape, but this resulted in all uncaught errors coming back as "uncaught error", needless to say that wasn't very useful. ;)

It's been intentional behavior: fatal errors are not recoverable and it's generally not safe to continue (even if the error is caused simply by an uncaught error), so it's not appropriate as a recovery mechanism for most applications (there are rare exceptions though) and the applications should really just use protected calls to wrap script logic which may throw errors. As such verbosity is less important than minimizing the potential for further errors before the user callback gets to handle the fatal error (e.g. it's not nice to segfault trying to format an error before making it to the user callback).
But since users often start developing without protected wrappers, they get fatal errors and instead of adding protected call wrappers they may get frustrated with the fatal errors... :o

One approach would be to leave the fatal error handling as is, but add a hook for handling uncaught errors. The hook could be an Ecmascript or Duktape/C function which would be given an uncaught error before the fatal error mechanism gets involved. That hook could then e.g. log the error to stderr before exiting, after which the fatal error handler would be called normally. (There's no option to not call the fatal error handler, because there's no error catcher and no way to unwind.)
While an uncaught error leads to a fatal error, there's not much risk in trying to format or print the error in this case (as opposed to e.g. when an assertion error happens).

For what it's worth, here is what's said in the API doc for duk_create_heap():
A fatal error handler is provided in fatal_handler. This handler is called in unrecoverable error situations such as uncaught errors, out-of-memory errors not resolved by garbage collection, etc. A caller SHOULD implement a fatal error handler in most applications. If not given, a default fatal error handler is used. The default handler ultimately calls ANSI C abort(), which may not always be the preferred action.

As it's specifically mentioned that the fatal error handler is called in case of uncaught errors, and the general recommendation is to implement one except in specific circumstances, I think it's going to be most intuitive for new users to assume said handler is the proper place to deal with uncaught errors.  I know that was the case for me, I didn't discover the protected calls until quite a bit later.

+1 for the hook function


Company Page - Pagination in add service and required action button in project view By Suresh sir

Describe the bug

Need project status change option

there is 23 service but only showing 20 service

Problem Statement?
Kisko chahiye?
Any other Option?

dont merge 2 different issue in 1
23 service ke jagah 20 aa rahi he? ye kya he?
dont just add issue, check background, if any details,

Create Different issues with details

https://www.berrycast.com/conversations/8a6a84b0-502f-5559-b857-13b65fe806fc

Error Details-
Multiple Company Selection  -> Add Service -> gives only 1st Page , 20 Services,
paging required

Initiated By @sureshsirvi
Verified by @Avinash-ecb

When adding multiple service there is showing all service which are recurring and non recurring , recurring service should be there and user can not change that "have recurring filter"


Remove the remaining unnecessary closures in the src/core/primitives.js file

/botio test

/botio unittest


fix(array): make Array.from work with mapping functions

Issue referenced at:
https://github.com/aurelia/polyfills/issues/4#issuecomment-246473333
Previous behavior resulted in errors thrown when passing a mapping
function to the polyfill Array.from
This allows you to properly allow a mapping function and a “thisArg” to
follow spec listed
(https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/from)

A quick look at the code, it seems to me that the bug was on line 32, where the iter parameter was missing.
L32: iterCall(mapfn, step.value, index)
should have been
iterCall(iterator, mapfn, step.value, index)

Thanks for clearing things up. That was one of the things that was wrong, but we also needed to return the result of the map function too, otherwise it'll set each result of the array to be undefined. I'll get that update in real quick.

How do those changes look @jods4?

You are right, the return was missing as well.
Looks good to me now.

@EisenbergEffect, can we get this merged, pending your review?

Ok, it's merged. Give me another few minutes and I'll have a patch release out.


Fix to use host specific password/path for the principal retrieval for customer with host specific keystore passwords

Description
Fixes the following customer scenario - before executing the playbook the customer creates key/trust stores for brokers and passes the passwords for them via set_fact to the cp-ansible playbook. These passwords are per-host, so when the DN is extracted from the cert to be added to super.users the current implementation uses the password from the host that is running (run_once) the extraction. This causes the extraction on the other hosts to fail and the super.users are not filled correctly essentially User:<DN of host running the task,User:;User:; etc.  This fix takes the password from each individual hosts facts and runs the DN extraction.
Fixes # (issue)
Incorrect password used for the keytool execution when per-host certs with cert specific passwords are used.
Type of change

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

How Has This Been Tested?
Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration
Tested normal scenario where kafka_broker_keystore_pass is provided in the hosts.yml as a group variable. Also tested with the following setup (simulates what the customer does):
Wrapper playbook:
- name: playbook to set per host passwords
   import_playbook: all_include.yml
- name: run cp-ansible
   import_playbook: all.yml

The all_include.yml just sets the fact
- hosts: kafka_broker
   tasks:
   - name: set per host password
      set_fact:
        kafka_broker_keystore_storepass_test: "test1234-{{ inventory_hostname }}"
        cacheable: yes

In the inventory file set the group variable
kafka_broker_keystore_storepass: "{{ kafka_broker_keystore_storepass_test }}"

Then just execute. Verified with debug output that the execution uses the host-specific variable when executed.
Test Configuration:
RBAC, mTLS
Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] Any dependent changes have been merged and published in downstream modules
[ ] Any variable changes have been validated to be backwards compatible

@thecrazymonkey Thank you for the contribution, this looks great.  Can you fill in more details in the description and provide a test to validate this works?

Hi Justin, added some more detail, sorry for missing it the first time. Let me know if more is needed, please. Thanks!

Hello @thecrazymonkey It seems these changes are merged in 7.1.x branch with this PR Do you still want these to be part of 6.2.x  as well ?


keep the raw value to allow the override on every request

Last PR still has a bug
Signed-off-by: Krasi Georgiev<EMAIL_ADDRESS>

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 8 (0.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.009%) to 6.017%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pkg/tracker/index/index.go
0
8
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.009%

Covered Lines:
118

Relevant Lines:
1961

💛  - Coveralls


Support logging in json format

In our clusters we use fluentbit to send logs to CloudWatch, which then allows us to use CloudWatch Logs Insights to query for patterns, either for one-time debugging or auditing, or in some cases, even creating Grafana dashboards. Having the option to output the logs in json format allows for improved searching on specific fields or patterns.
Taking this on would likely mean using a different logging package, since it doesn't look like glog supports this. I've had good success in the past with logrus, but it looks like even they now recommend some other improved packages, such as: Zerolog, Zap, and Apex.

Yes, glog was used AFAIR because it was already there in the kubernetes client. What would you choose?

Of those that I've used, zerolog seemed to me the right combination of usability and performance

Would you like to create a PR for this? :)

Sure :) Would you like it to be json as the default, or as an option?

Makes sense, as it will run on Kubernetes. If it makes sense to you as the default as well then I think there's no need to overthink it ;)


Formatting on save: script type=module broken

We have written the needed data into your cl
Issue Type: Bug

when saving the formatting on save kicks in (great feature), but inside the script tag it puts everything on a single line. When the line is full the formatter creates a hard break which causes all kinds of issues, e.g. incomplete strings, etc. It also leads to having single line comments suddenly afects a whole lot more code.
result of step see below:

I currently can workaround by saving without formatting, but this needs to be fixed.
 Please paste.

/needsMoreInfo
Is it reproducible with all extensions disabled? Especially vscode-JS-CSS-HTML-formatter
You can try this with F1 and >Developer: Reload Window With Extensions Disabled
If it's still reproducible with all extensions disabled please show code before and after that is wrapped in code block ``` so it would be easier to reproduce. Example:
```
code
```

Thank you @IllusionMH
I reloaded without extensions and behaviour was fine! I will test all extensions individually to find the culprint and open a defect for this then.
Can be closed

Most likely vscode-JS-CSS-HTML-formatter but if it's not his extension you can try F1 >Help: Start Extension Bisect that should simplify search.
See https://code.visualstudio.com/updates/v1_52#_troubleshooting-extension-bisect


Automate initialization of newly deployed Vault clusters

This is based on https://github.com/mitodl/ol-infrastructure/pull/408 which needs to merge first.
Once we deploy a new Vault cluster into an autoscaling group we need to be able to initialize it and set up the initial set of authentication and audit backends. This adds a script that will initialize the cluster, log the recovery keys, and provision an account to be used by Pulumi for managing the cluster going forward.

Addresses https://github.com/mitodl/ol-infrastructure/issues/328

👍


Improving lambda Cold start

Is your feature request related to a problem? Please describe.
A Lambda cold start happens when a new instance of a Lambda function must be created and initialized. The cold start refers to the delay between invocation and runtime created by the initialization process.
New instances needs to be initialized whenever other instances have expired due to inactivity or when there are more invocations than active instances. Cold starts are an inherent problem with Lambda functions because it is not possible to keep lambda initialized forever.
The OpenTelemetry SDK was not created with Lambda functions in mind. If you use OpenTelemetry inside a lambda function, the overhead of initializing the SDK and optionally auto instrumenting the application code adds up in the cold start time. This is specially painful for users because this inserts high latency in their application and increases the cost of running the lambdas.
Describe the solution you'd like
This proposal will tackle the cold start time of the OpenTelemetry lambda layers with the following plan:
Plan:

Continuously measure the cold start time of the layers with each release. This will help catching regressions in performance and also show trends and where we should invest our time.
Profile each layer to identify where all this time is spent on in the code.
Propose optimizations in the initialization of the SDK in each layer: Using the profiling information from the previous step, look for low hanging fruits and also more complex refactoring that will improve the performance.

Methodology for measuring the cold startup:

Measure the cold start time for lambdas with and without the layers for each supported layer.

Create a sample application and deploy to a lambda function
Generate load for this sample application
Vary a parameter in the lambda function that will force the lambda to be recreated.
Parse the logs of the lambda function with the following query:

filter @type="REPORT"
| filter ispresent(@initDuration)
| stats count(@initDuration) as coldStartCount, pct(@initDuration, 50) as p50Init, pct(@initDuration, 90) as p90Init, pct(@initDuration, 99) as p99Init group by @log

Methodology for profiling the lambda functions:

TBD - We will need to

Additional context
References
https://github.com/open-telemetry/opentelemetry-lambda/issues/263

Cool! Glad we're addressing this issue. :-)

Hi, is there any timeline or proposals in planning or in place to address the metrics on cold start being recorded?

Just ICYMI, Java agent has fast startup support for Lambda: https://github.com/open-telemetry/opentelemetry-lambda/tree/main/java#fast-startup-for-java-agent


Can't Uninstall

Ran into errors when trying to use then when attempting to disable I just get a continuous error everytime i attempt to restart

Did you tried to remove/delete %appdata%\XIVLauncher\installedPlugins\Divination.DiscordIntegration folder manually?
I think you can uninstall this plugin by this step.
And for fixing the error, developer needs some logs to investigate why that error happened.
So if you can, please upload %appdata%\XIVLauncher\dalamud.log!


Broken pseudo selector body if it contains more than one style attribute

Using pseudo selectors we found an issue. Whenever we have more than one style attribute inside the pseudo selector its not rendered properly.
e.g.
foo: {
  ':hover': {
    color: '#ffffff',
    display: 'block',
    height: 0
  }
}

leads to:
.foo:hover {
    color: #ffffff;
    , display: block;
    , height: 0;
}

I also provided #20 which fixes this.

Thanks for reporting this issue and providing your pull request!
I just published version 1.1.1 which includes your fix.
Cheers

@davidspinat thanks a lot for beeing so responsive :)


aconcli: Issues in command line syntax/flags

This issue tracks several (minor) problems in aconcli command line syntax/flags. These problems should be easy to fix.

[ ] generate errs on existing FS layer symlinks
[ ] generate should take output file name as a parameter and docker image/tag by a flag - as support to container tools other than docker may be added in future.
[ ] export - --ignoresig doesn't seem to have been implemented. Also, do we have to check signature when exporting an image? Can we just export whatever available?
[ ] ls and prune err on empty ACON image repo
[ ] report missing flag/option to request a quote instead of a report
[ ] report should generate nonce at random if not specified by the user
[ ] stop and shutdown should have command line syntax/flags consistent with other commands - i.e., -c for specifying ACON TD/VM and -e for specifying container.
[ ] sign errs on re-signing, also due to existing symlinks like in the generate command.

@xxu36, I found one more bug in sign that is very similar to generate - it errs on existing symlink. I've appended that the the tasklist above.

I'll look at ls and prune issues this week. @xxu36 please have a look at other task items.

ls and prune issues are fixed in #52

Let's implement --ignoresig now. For shutdown and stop command syntax, we may have to change if unix socket forwarding can work, so postpone to next release.


More easily navigate between Cluster Explorer, cluster detail and cluster config

I had some feedback based on my experience of stepping through every 2.6 feature while rewriting every set of instructions in the docs for Rancher 2.6. Here are two things I thought would have helped make it easier to navigate.
One thing that threw me off when I first tried documenting how to configure clusters was that I first tried to go to the cluster explorer page because I expected that to be a source of truth for everything related to that cluster. I also did the opposite, clicking the name of the cluster in the cluster list view when I wanted cluster explorer, then finding myself in the detail page, then having to go back. I found it counterintuitive that to edit the configuration of the cluster, you have to go to the global view first instead of being able to stay within a cluster section. I don't think you should have to go up in the hierarchy to make a lateral move.
There should be a link to Cluster Explorer from the cluster detail page and vice versa.
Edit: Removed the part that was already fixed.

I also don't like that the tabs on the cluster detail page rerender (visibly disappearing and reappearing) when you navigate between them. The tabs should stay there without rerendering while the content under them changes

.

I also don't like that the tabs on the cluster detail page rerender (visibly disappearing and reappearing) when you navigate between them. The tabs should stay there without rerendering while the content under them changes

I just want to add that I came across this the other day and it's a very confusing experience

Yeah, it's confusing, especially when you navigate from YAML to Config. In that case the tabs linger for a moment, then disappear entirely.

We should add the left action menu from the cluster management page to each cluster in the dashboard page.


Redo: Find correct path of libcudnn.dylib on macOS.

I just found that commit a62c532 overwrote my change in #5944.

Can one of the admins verify this patch?


Documenting ChakraCore's internals

As part of improving the experience for contributors we should produce helpful documentation covering the logic of ChakraCore's internals. Including what the key data types are in each area and what key logical processes are used internally:
It would be good if multiple people could contribute to this, we'll need to think about exactly how it should all hang together but here's a rough list of the things we should document:

[ ] Process overview from script file -> execution with notes on where different aspects of the codebase are involved
[ ] Parser
[ ] Bytecode emitter
[ ] Type system
[ ] Interpretor
[ ] JS Library
[ ] JsBuiltins
[ ] INTL
[ ] GarbageCollector/Recycler
[ ] Jit
[ ] Internal Data Structures
[ ] RegEx parsing and matching

Not detailed enough at all, but here's some draft introductory documentation on the Regex parsing and matching system: https://gist.github.com/rhuanjl/913e7bf4993cc8f07c04cf6de154f572


Upgrade dependencies

We should upgrade dropwizard and jackson. Plus a few more dependencies like maven plugins.
Upgrading just jackson is not sufficient: https://github.com/graphhopper/graphhopper/commit/dffc748595755f6b10cb445c89f68a193747b179 as it is better to use a more recent dropwizard (1.3.12) that uses the latest jackson version.
The problem of 1.3.12 is that the dropwizard-configurable-assets-bundle in web/ is not supported, see: https://github.com/dropwizard-bundles/dropwizard-configurable-assets-bundle/issues/36 but the tests are passing fine with 1.3.12 too so we just keep the old version for now but need to keep an eye on this everytime we upgrade dropwizard.

Will close here until something is broken due to this.


[dotnet][java][js][py][rb][rust] Update firefox_standalone Docker digest to 39f05e4

This PR contains the following updates:

Package
Type
Update
Change

firefox_standalone
oci_pull
digest
b6d8279 -> 39f05e4

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

Edited/Blocked Notification
Renovate will not automatically rebase this PR, because it does not recognize the last commit author and assumes somebody else may have edited the PR.
You can manually request rebase by checking the rebase/retry box above.
⚠ Warning: custom changes will be lost.


Running with Scissors in the Matrix results discrepancy

Hi,
The results of the OPRE paper and MeltingPot paper in the environment Running with the Scissors in the matrix (which is almost the same in both the papers as discussed in #68) do not match.
OPRE Paper self-play results:

MeltingPot reported scores:
Note: All Meltingpot agents are trained using self-play as mentioned in the paper.

The only difference based on my understanding is that the OPRE paper initialized 6 different agents (RWS has only 2 agents in the environment) with different seeds whereas the MeltinPot paper initialized 2 agents (RWS has only 2 agents in the environment)
Is there anything I'm missing here?
Could you please comment on what could be the reason behind this discrepancy?
Thanks
Kinal

Hi!
Another difference is that in OPRE paper the overall reward scale of the game. The Melting Pot version has rewards scaled down by 0.01 compared to the original (OPRE paper) version. Essentially, to compare one has to multiply Melting Pot results by 100

Oh, I somehow missed that.
Thank you for clarifying.
With this scaling, the scores of meltingpot paper are ~5 by averaging over sc2 (pure rock), sc3(pure paper), sc4(pure scissors).

I think this one is on us, I just looked at the melting pot paper and it's not that easy to get it (you have to pay attention to the matrix of RWS game). We should have been more explicit.

With this scaling, the scores of meltingpot paper are ~5 by averaging over sc2 (pure rock), sc3(pure paper), sc4(pure scissors).

The corresponding score in the MP paper is against SC 1, which is a pure bot (r/s/p) chosen at random vs the agent. But yes, you could also average the SC 2-4.
Overall, my explanation of the difference between OPRE paper and Melting Pot results is, as you correctly pointed out, due to the training protocol. Having 6 agents playing each other helps with i) exploration ii) not getting stuck in local minima via diversity (having to respond to 5 different agents at all times instead of just one).

Thank you so much for this perspective.

Hey @vezhnick,
I had a question regarding the training of OPRE.
It is mentioned in the OPRE paper that all 6 agents learn simultaneously.
Considering the RWS environment with 2 agents, I believe the experience collection loop samples 2 agents from the set of 6 and uses those 2 to generate actions/experiences.
In the learner, is it that for both the players from the environment, each agent is updated playing against itself as follows

agent 0 vs agent 0
agent 1 vs agent 1
and so on.

Or is it that 3 random pairs are made for training

agent 1 vs agent 3
agent 2 vs agent 0
agent 4 vs agent 5

Or is there any other strategy to train the entire population on the same data?
Thanks

Hey there!
It is the second version, meaning they are mixed at random. We ran several hundreds of matches between random pairs of agents in parallel, collecting the data and shipping it to learners in distributed fashion.

As you are running experience collection in parallel between random pairs, I believe the process of learning and experience collection is asynchronous.

So, in the learner, is the information about which experience is between which pair of agents passed through the replay buffer? Or is the paring in learner independent of how the experience was collected?
E.g.:

Process 1 experience agent 3 vs agent 0
Process 2 experience agent 1 vs agent 4

And as the learner is getting samples from both of these matches, how to manage matching in the learner?

I also had more questions regarding the OPRE paper. Hope you don't mind me asking them here.

In section 5.4 ablations, a variant called PureMix is mentioned, which is used to check the effectiveness of the mixture model. Is the PureMix variant just conditioning the final policy on the latent dimension $z$, whereas the OPRE uses a mixture?
Based on my understanding, the Value function and $\mu$ and $\pi$ policy for PureMix would be:
$V = c(x_e)$ and $\pi = \eta(a|x_e)$
where $x_e = concatenate([z,x_{\le t}])$ and $z = q(x'_t)$ a categorical distribution predicted using concealed information. And similarly, for $\mu$

Hey! Sorry for a late responce.

So, in the learner, is the information about which experience is between which pair of agents passed through the replay buffer? Or is the paring in learner independent of how the experience was collected?

We did not use replay at all, it was all done in online fasion. You could think of it though as a replay buffer, where each trajectory is sampled only once and buffer is a FIFO.

And as the learner is getting samples from both of these matches, how to manage matching in the learner?

Actually, in your example the experience from the first match would only go to the 3rd and 0th learner, while the second would go to 1st and 4th. I'm not sure I get the question right though.

In section 5.4 ablations, a variant called PureMix is mentioned...

Pure mix is very simple, it is the same as the basic baseline (advantage actor critic with V-trace), but policy is represented as a mixture. It is a lot closer to baseline than to OPRE. Sorry if that wasn't very clear.
The version you are suggesting would be a good ablation as well.
P.S. feel free to shoot me an email

Thank you for these details.
I'll email you with more questions and details about my work.
I believe your email-id is: vezhnick [at] deepmind [dot] com
Please correct me if I'm wrong.

Closing as I think this is resolved.

Hey @vezhnick,
I have mailed you a few more queries regarding OPRE on "vezhnick [at] deepmind [dot] com". I hope you have received it.


add doc for startup of eBPF program

What type of PR is this?

What this PR does / why we need it:
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

IMO, the calling flow can be changed frequently, can we write a blog under https://kmesh.net/en/blog/


Wrong argument order to ballerina command

Description:
In generated Docker file, order of arguments to ballerina command is wrong.
Currently generated Docker CMD
CMD ballerina run hello_service.balx --config /home/ballerina/conf/twitter.toml
This should be (in 0.982.0)
CMD ballerina run --config /home/ballerina/conf/twitter.toml hello_service.balx
Affected Product Version:
0.982
Steps to reproduce:
Follow docker section of Ballerina quick-tour.

This issue is fixed with https://github.com/ballerinax/docker/pull/139, hence closing this issue.


export_fig -clipboard fails in compiled mode

When I use the export_fig -clipboard option in standard Matlab all works fine.  Yet, when I compile it then it makes problems:
Warning: export_fig -clipboard output failed: Undefined function 'ImageSelection' for input arguments of type 'java.awt.image.BufferedImage'.

In export_fig (line 803)

You need to add the ImageSelection.class file to your compiler project's setup as a static resource. This file is located in the main export_fig code folder.


GA4GHTT-270: Merge changes from master (241, 247, 248) and porting 248 for 4.4

Squashed and merged 241, 247 and 248 to dev/vcf4.4.
v4.4 Ragel and test files updated based on 248.
This is made on top of pr 249.

23e909d is PR #249. I expect it to be removed / ignored as that gets merged.


[RN 0.74.2] pod install fails on 2.3.0

After adding the package to react-native app pod install fails with the codegen error:
[!] Invalid Podfilefile: [!] /Users/user/.nvm/versions/node/v18.20.3/bin/node ./../node_modules/react-native/scripts/generate-codegen-artifacts.js -p /Users/user/rn-app/ios/.. -o /Users/user/rn-app/ios -t ios
Someone faced this problem?

dupe of https://github.com/mkuczera/react-native-haptic-feedback/issues/127

Fixed by #129

Merged. Will just invest a little bit time to have a proper testapplication and pipelines to avoid future fails.

i'm seeing this problem as well when using 2.3.0. theres multiple RN packages thats facing this exact issue.

Can be verified in "2.3.1-next.0".
npmjs.com/package/react-native-haptic-feedback/v/2.3.1-next.0
Would be awesome to have feedback that everything works fine before releasing an official version :)

@mkuczera Confirmed that 2.3.1-next.0 installs, builds, runs, and works successfully for me on my physical iPhone device. My app doesn't use android so I haven't tested that.
Previously "^2.2.0" (not sure what version it was resolving too) had recently started failing on "pod install".

Installation worked, but got a build error on iOS:
node_modules/react-native-haptic-feedback/ios/RNHapticFeedback/RNHapticFeedback.h:3:9 'RNHapticFeedbackSpec.h' file not found

Thanks for the feedback. I will check RNHapticFeedbackSpec. This was adjusted 17 months ago and didn´t remove it. oO

Thank you all. Version 2.3.1 is release. @villemuittari just digged a little bit, but shouldn´t be crucial for any build process of your app. Also tried various configurations. But maybe you can point me to your setup where this error occured (added the spec file, but was working fine also without it)


Fix .NET Core support on macOS.

Now if you try to build .NET Core app on macOS you'll get an exception:
System.DllNotFoundException: Unable to load DLL 'System.Security.Cryptography.Native': The specified module could not be found.
 (Exception from HRESULT: 0x8007007E)
   at Interop.CryptoInitializer.EnsureOpenSslInitialized()
   at Interop.CryptoInitializer..cctor()
   --- End of inner exception stack trace ---
   at Interop.Crypto..cctor()
   --- End of inner exception stack trace ---
   at Interop.Crypto.GetRandomBytes(Byte* buf, Int32 num)
   at System.IO.Path.GetCryptoRandomBytes(Byte* bytes, Int32 byteCount)
   at System.IO.Path.GetRandomFileName()
   at Microsoft.DotNet.InternalAbstractions.TemporaryDirectory..ctor()
   at Microsoft.Extensions.EnvironmentAbstractions.DirectoryWrapper.CreateTemporaryDirectory()
   at Microsoft.DotNet.Configurer.NuGetPackagesArchiver..ctor()
   at Microsoft.DotNet.Cli.Program.ConfigureDotNetForFirstTimeUse(INuGetCacheSentinel nugetCacheSentinel)
   at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, ITelemetry telemetryClient)
   at Microsoft.DotNet.Cli.Program.Main(String[] args)

It relates to known issue.
To solve this problem it's necessary to do the following:
~$ mkdir -p /usr/local/lib
~$ ln -s /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib /usr/local/lib/
~$ ln -s /usr/local/opt/openssl/lib/libssl.1.0.0.dylib /usr/local/lib/

Moreover, official .NET Core installation guide tells that you should run commands desribed above before using .NET Core on macOS.
So, this PR adds this commands which will be perform on macOS before installing .NET Core.

I found out that an instruction brew link --force openssl is depricated and  instructions
~$ mkdir -p /usr/local/lib
~$ ln -s /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib /usr/local/lib/
~$ ln -s /usr/local/opt/openssl/lib/libssl.1.0.0.dylib /usr/local/lib/

are just instead of brew link --force openssl.
This problem is discussed in this PR: Homebrew/brew#597.
So I added one more commit to remove this instruction.

Please set up a repo on https://staging.travis-ci.org so that we can coordinate testing this PR.

@BanzaiMan I set up aidmsu/Hangfire repo. It has two PR: PR#2 and PR#3. PR#3 contains commands to link to openssl libs and build is successful on macOS . PR#2  doesn't contain the commands and build failed on macOS.

Because it is not deployed. I'll let you know when.

I plan to deploy this to staging around 15:00 UTC tomorrow.

@BanzaiMan Thanks! I'm looking forward to it.

@aidmsu This has been deployed to staging.

@BanzaiMan I restarted build of PR#2 and it passed on macOS now!


TypeError: init() got an unexpected keyword argument 'name' while loading saved model in Keras 0.3.2, python 2.7

While loding pre-trained model, I am getting the following error. Could someone please help me to fix it? For the moment I would like to stick with Keras 0.3.2.
In [10]: g = model_from_config(json_dict)
TypeError                                 Traceback (most recent call last)
 in ()
----> 1 g = model_from_config(json_dict)
.local/lib/python2.7/site-packages/Keras-0.3.2-py2.7.egg/keras/models.pyc in model_from_config(config, custom_objects)
175
176     # Create a container then set class to appropriate model
--> 177     model = container_from_config(config, custom_objects=custom_objects)
178     if model_name == 'Graph':
179         model.class = Graph
local/lib/python2.7/site-packages/Keras-0.3.2-py2.7.egg/keras/utils/layer_utils.pyc in container_from_config(original_layer_dict, custom_objects)
55         nodes = layer_dict.get('node_config')
56         for node in nodes:
---> 57             layer = container_from_config(layer_dict['nodes'].get(node['name']))
58             node['layer'] = layer
59             graph_layer.add_node(**node)
local/lib/python2.7/site-packages/Keras-0.3.2-py2.7.egg/keras/utils/layer_utils.pyc in container_from_config(original_layer_dict, custom_objects)
90         if 'custom_name' in layer_dict:
91             layer_dict['name'] = layer_dict.pop('custom_name')
---> 92         base_layer = get_layer(name, layer_dict)
93         return base_layer
94
.local/lib/python2.7/site-packages/Keras-0.3.2-py2.7.egg/keras/utils/layer_utils.pyc in get_layer(identifier, kwargs)
156 def get_layer(identifier, kwargs=None):
157     return get_from_module(identifier, globals(), 'layer',
--> 158                            instantiate=True, kwargs=kwargs)
.local/lib/python2.7/site-packages/Keras-0.3.2-py2.7.egg/keras/utils/generic_utils.pyc in get_from_module(identifier, module_params, module_name, instantiate, kwargs)
16             return res()
17         elif instantiate and kwargs:
---> 18             return res(**kwargs)
19         else:
20             return res
TypeError: init() got an unexpected keyword argument 'name'
Please make sure that the boxes below are checked before you submit your issue. Thank you!

[  ] Check that you are up-to-date with the master branch of Keras. You can update with:
pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps

[x] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

[x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

I also have this problem.

I ran into similar issue while loading model saved with python 3.4 into python2.7. No issue loading the model with python3.4.


Add Ankur Gupta to Contributors list

@ankurg22, I'm quite elated about your PR. I wanna evolve this project to addresses various problems faced by first time contributors. I'd love to learn about your journey in open source community, the problems, pain points you had etc.
Could you explain how you felt when you went through the tutorial, made a PR and learned that I accepted it?


Temporarily remove _header file

Third party icons are not loading, similarly the newsletter archive styling etc. is broken, moving forward, let's copy the contents in this repository so that everything is hosted from one location.
Let's make a copy of every icon that we're using + the stylesheets and re-enable this instead? @ayodejidev


Reproducing RotMNIST results

Hi,
Thank you for releasing this code, and I quite liked the paper.
I've had some difficulty getting the RotMNIST experiments working. From ./experiments/rotmnist I'm simply running python runner.py with the default argparse setting. I've slightly modified this file only to do a train/val split and print the train/val accuracies (exact code here).
I'm getting poor training and val accuracies that are not improving after the first few epochs (see below)--the numbers are below the same architecture trained naively (without any augmentation). Should I be using different hparams from the default? Or does it just require more epochs?
Any advice or tips would be greatly appreciated!
Thanks!

Other info: python 3.7.8, torch 1.7.0, cuda 10.1. System is Ubuntu 16.04.

Hi Allan,
Thanks for the heads up - for me it looks the primary issue is that the default learning rate is just too high. If I drop the learning rate to 0.01 I am getting training accuracy of 0.85-0.9 by about the third epoch.
The network in that file is also a very small network just to keep things easy to run, if you want to get better accuracy you can swap the model definition line in the runner.py file with:
net = models.layer13s(in_channels=1,num_targets=10, k=32)
(You can also push k up to 128, but it will take a bit longer to train). For me the layer13s gave >90% train accuracy after 3 epochs.
Let me know if that helps!
Best,
Greg

On Dec 17, 2020, at 4:17 PM, Allan Zhou<EMAIL_ADDRESS>wrote:
Hi,
Thank you for releasing this code, and I quite liked the paper.
I've had some difficulty getting the RotMNIST experiments working. From ./experiments/rotmnist I'm simply running python runner.py with the default argparse setting. I've slightly modified this file only to do a train/val split and print the train/val accuracies (exact code here https://github.com/g-benton/learning-invariances/compare/master...AllanYangZhou:master).
I'm getting poor training and val accuracies that are not improving after the first few epochs (see below)--the numbers are below the same architecture trained naively (without any augmentation). Should I be using different hparams from the default? Or does it just require more epochs?
Any advice or tips would be greatly appreciated!
Thanks!
https://user-images.githubusercontent.com/8732147/102544775-4c4de980-4083-11eb-95f9-f3c81f8e80ad.png
Other info: python 3.7.8, torch 1.7.0, cuda 10.1. System is Ubuntu 16.04.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/g-benton/learning-invariances/issues/1, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG7A6U5GEYNLAB4CYHUGLDDSVJYOXANCNFSM4VAICAKA.

Yes, adjusting the learning rate did the trick. I probably should've tried that earlier--I'll also try increasing the architecture size. Thank you for the prompt reply!
Best,
Allan


[SMALLFIX] Typo in cp command line description.

Can one of the admins verify this patch?

ok to test

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/10682/
Test PASSed.

LGTM, thanks for the fix!


add CreatePalette stub

stub version of #49, will rebase that one on this after merge

Aw bummer, I can see that the gdi32.dll conflict is gonna be a repeated annoyance.  What to do, hrmm....

(change looks good otherwise)

rebased/merged manually

Yeah, it's a bit annoying 😅
I've been submitting just one PR per dll file earlier, but missed it here. Thanks for merging manually 🙏


Extend transactionAPI to provide a payment method details

Problem
TransactionItem doesnt have any fields that would allow to store the details related to used payment method. Like: payment method type, last 4 digits etc. Currently payment app needs to use metadata to store such data.

1. General Assumptions
TBD

Acceptance Criteria

TBD

[ ]
[ ]

2. API changes
Mutations
TBD
Types
TBD
3. Database changes
TBD

4. UML diagrams

5. To Do list

### Tasks
- [ ] Add a draft title or issue reference here

6. To Test

[ ]
[ ]

As of now I'm using order meta data to store those details. I would be great if we have that feature in transaction its self.


Absolute refs not always resolving absolutely

Commit 3eb313fe introduced a logic flaw: When an absolute ref does not resolve to a page, the ref is then resolved as a relative ref or an "ambiguous ref" depending on the circumstances. Expected behavior is that an absolute ref, identified by a leading /, is always and only resolved as such.
I submitted PR #4970 with tests that show this behavior. The relevant tests are tagged "ISSUE #4969". The original logic as submitted in #4796 passes these tests.

By anyone who's used a file system or the web.
What's the point of an absolute path if it also has relative semantics? File systems and the web both have clear cut absolute and relative path semantics for a reason. That Hugo, which essentially takes one (the file system) and translates to the other (the web), doesn't follow the same rules seems bad design to me. The accommodations for language independence and even making the file extension optional I get, but allowing /a/b.md to resolve to /x/y/a/b.md doesn't make any sense to me.
I think this stuff matters a lot, and will cause Hugo users pain and confusion if left as-is. It's important to understand why Hugo is popular, and not assume its popularity is a validation of all of its design choices. What's the point of being beta, of being pre v1.0 if it isn't to get the model right before it is set in stone?
Anyway, you're boss, the BD. This is just my small opinion.

Anyway, you're boss, the BD. This is just my small opinion.

I will look at this.
But note that when I tested your PR on 5 of the sites I have floating around, 4 of them failed miserable with "ambiguous errors" that would be really hard to explain to the common man. The low-estimate of added support work (questions, issues etc.) is 50 hours.
If people say:
{{ .Site.GetPage "page" "blog/mypost.md" }}

The above is a very common construct. Is that a relative path?
All of this would be easier to handle for me if you had just kept the old methods/tests and created some adapter functions.

@vassudanagunta I have added two commits (the last one is mainly a test clarification), but the first one makes sure that absolute references isn't tried as relative if there is a page context (i.e. you come from .Page.GetPage or ref/relref). When you come from .Site.GetPage relative/absolute does not make much difference to me (they are all relative to the content root).

@bep, cool, that fixes the worst inconsistency (line 218 of the test, enabled test with #4971). There is one other that affects page relative queries, but I get that it can't be fixed for backwards compatibility reasons. Same for the other commented out tests.
I'm sorry i didn't mean to make your life harder. I spent a lot of time on this myself. Two chefs in the kitchen usually a problem... I'll step out. I don't do well just pealing potatoes. ;)


core: Tokio and qdrant tweaks

Shared qdrant connection
Chunked upsertion to qdrant
Remove an std:thread sleep

Fully tested locally (including JS timeout)

cc @lasryaric @fontanierh


[Bug]: When using vllm in a Ray actor, the error "No CUDA GPUs are available" occurs.

Your current environment
PyTorch version: 2.3.0a0+40ec155e58.nv24.03
Is debug build: False
CUDA used to build PyTorch: 12.4
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.4 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: Could not collect
CMake version: version 3.28.3
Libc version: glibc-2.35

Python version: 3.10.12 (main, Mar 22 2024, 16:50:05) [GCC 11.4.0] (64-bit runtime)
Python platform: Linux-4.19.91-014-kangaroo.<IP_ADDRESS>c249cdaf.x86_64-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 12.4.99
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration:
GPU 0: NVIDIA H800
GPU 1: NVIDIA H800

Nvidia driver version: 525.105.17
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.9.0.0
/usr/lib/x86_64-linux-gnu/libcudnn_adv.so.9.0.0
/usr/lib/x86_64-linux-gnu/libcudnn_cnn.so.9.0.0
/usr/lib/x86_64-linux-gnu/libcudnn_engines_precompiled.so.9.0.0
/usr/lib/x86_64-linux-gnu/libcudnn_engines_runtime_compiled.so.9.0.0
/usr/lib/x86_64-linux-gnu/libcudnn_graph.so.9.0.0
/usr/lib/x86_64-linux-gnu/libcudnn_heuristic.so.9.0.0
/usr/lib/x86_64-linux-gnu/libcudnn_ops.so.9.0.0
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture:                    x86_64
CPU op-mode(s):                  32-bit, 64-bit
Address sizes:                   52 bits physical, 57 bits virtual
Byte Order:                      Little Endian
CPU(s):                          8
On-line CPU(s) list:             0-7
Vendor ID:                       GenuineIntel
Model name:                      Intel(R) Xeon(R) Processor
CPU family:                      6
Model:                           143
Thread(s) per core:              1
Core(s) per socket:              8
Socket(s):                       1
Stepping:                        8
BogoMIPS:                        5200.00
Flags:                           fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap avx512ifma clflushopt clwb avx512cd sha_ni avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves avx512_bf16 wbnoinvd avx512vbmi umip pku waitpkg avx512_vbmi2 gfni vaes vpclmulqdq avx512_vnni avx512_bitalg avx512_vpopcntdq rdpid cldemote movdiri movdir64b fsrm md_clear arch_capabilities
Virtualization:                  VT-x
Hypervisor vendor:               KVM
Virtualization type:             full
L1d cache:                       384 KiB (8 instances)
L1i cache:                       256 KiB (8 instances)
L2 cache:                        16 MiB (8 instances)
L3 cache:                        97.5 MiB (1 instance)
NUMA node(s):                    1
NUMA node0 CPU(s):               0-7
Vulnerability Itlb multihit:     Not affected
Vulnerability L1tf:              Not affected
Vulnerability Mds:               Not affected
Vulnerability Meltdown:          Not affected
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1:        Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers
Vulnerability Spectre v2:        Vulnerable, IBPB: disabled, STIBP: disabled
Vulnerability Srbds:             Not affected
Vulnerability Tsx async abort:   Not affected

Versions of relevant libraries:
[pip3] numpy==1.24.4
[pip3] onnx==1.15.0rc2
[pip3] optree==0.10.0
[pip3] pytorch-quantization==2.1.2
[pip3] pytorch-triton==2.2.0+e28a256d7
[pip3] torch==2.3.0a0+40ec155e58.nv24.3
[pip3] torch-tensorrt==2.3.0a0
[pip3] torchdata==0.7.1a0
[pip3] torchtext==0.17.0a0
[pip3] torchtyping==0.1.4
[pip3] torchvision==0.18.0a0
[pip3] transformers==4.43.2
[conda] Could not collect
ROCM Version: Could not collect
Neuron SDK Version: N/A
vLLM Version: 0.5.2
vLLM Build Flags:
CUDA Archs: 5.2 6.0 6.1 7.0 7.2 7.5 8.0 8.6 8.7 9.0+PTX; ROCm: Disabled; Neuron: Disabled
GPU Topology:
GPU0    GPU1    NIC0    NIC1    CPU Affinity    NUMA Affinity
GPU0     X      NV8     PHB     PHB     0-7             N/A
GPU1    NV8      X      PHB     PHB     0-7             N/A
NIC0    PHB     PHB      X      PHB
NIC1    PHB     PHB     PHB      X

Legend:

  X    = Self
  SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)
  NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node
  PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)
  PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)
  PIX  = Connection traversing at most a single PCIe bridge
  NV#  = Connection traversing a bonded set of # NVLinks

NIC Legend:

  NIC0: mlx5_0
  NIC1: mlx5_1

🐛 Describe the bug
We used VLLM to execute the qwen2 model with TP=4 in Ray Actor, but one of the four processes reported an error "No CUDA GPUs are available".
When executing vllm separately without Ray Actor wrapper, it can be executed normally.
[rank0]: ray.exceptions.ActorDiedError: The actor died because of an error raised in its creation task, ray::LLMRayActor.__init__() (pid=410475, ip=<IP_ADDRESS>, actor_id=90f87b7c95de4bb49deafe5c01000000, repr=<rlhf.vllm_generation.vllm_engine.LLMRayActor object at 0x7f8201a56080>)
[rank0]:   File "/cpfs/2926428ee2463e44/user/user1/rlhf/vllm_generation/vllm_engine.py", line 58, in __init__
[rank0]:     self.llm = vllm.LLM(*args, load_format=LoadFormat.AUTO, **kwargs)
[rank0]:   File "/usr/local/lib/python3.10/dist-packages/vllm/entrypoints/llm.py", line 156, in __init__
[rank0]:     self.llm_engine = LLMEngine.from_engine_args(
[rank0]:   File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 440, in from_engine_args
[rank0]:     engine = cls(
[rank0]:   File "/usr/local/lib/python3.10/dist-packages/vllm/engine/llm_engine.py", line 250, in __init__
[rank0]:     self.model_executor = executor_class(
[rank0]:   File "/usr/local/lib/python3.10/dist-packages/vllm/executor/distributed_gpu_executor.py", line 25, in __init__
[rank0]:     super().__init__(*args, **kwargs)
[rank0]:   File "/usr/local/lib/python3.10/dist-packages/vllm/executor/executor_base.py", line 47, in __init__
[rank0]:     self._init_executor()
[rank0]:   File "/usr/local/lib/python3.10/dist-packages/vllm/executor/ray_gpu_executor.py", line 61, in _init_executor
[rank0]:     self._init_workers_ray(placement_group)
[rank0]:   File "/usr/local/lib/python3.10/dist-packages/vllm/executor/ray_gpu_executor.py", line 233, in _init_workers_ray
[rank0]:     self._run_workers("init_device")
[rank0]:   File "/usr/local/lib/python3.10/dist-packages/vllm/executor/ray_gpu_executor.py", line 350, in _run_workers
[rank0]:     self.driver_worker.execute_method(method, *driver_args,
[rank0]:   File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker_base.py", line 383, in execute_method
[rank0]:     raise e
[rank0]:   File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker_base.py", line 374, in execute_method
[rank0]:     return executor(*args, **kwargs)
[rank0]:   File "/usr/local/lib/python3.10/dist-packages/vllm/worker/worker.py", line 124, in init_device
[rank0]:     torch.cuda.set_device(self.device)
[rank0]:   File "/usr/local/lib/python3.10/dist-packages/torch/cuda/__init__.py", line 424, in set_device
[rank0]:     torch._C._cuda_setDevice(device)
[rank0]:   File "/usr/local/lib/python3.10/dist-packages/torch/cuda/__init__.py", line 318, in _lazy_init
[rank0]:     torch._C._cuda_init()
[rank0]: RuntimeError: No CUDA GPUs are available

Hello @wyooyw.
I notice that your environment information shows that you have 2 GPUs:
GPU models and configuration:
GPU 0: NVIDIA H800
GPU 1: NVIDIA H800

Nvidia driver version: 525.105.17

Using 4-way tensor parallel with TP=4 will require 4 GPUs. So the error that "No CUDA GPUs are available" seems accurate.
When it worked without Ray do you also mean without TP=4?

sorry, it's a typo. we have two gpus and we use tp=2. There are more details: we have two test cases, one passed and another one failed with "No CUDA GPUs are available" as described above.
passed:

failed:
            from rlhf.vllm_generation.vlm.model_loader import MyMegatronLoader # this is my customized model loader
            self.llm = vllm.LLM(*args, load_format=AgiVlmMegatronLoader, **kwargs)

Ah, that makes more sense.
The second test fails even when it is the only test run in the Python session, right? If it only fails when running after the first test, then it may just be problem with the garbage collection not fully removing its usage of the GPUs.
If the second test fails on its own where the first one passes, then it seems that the issue would be coming from your customized model loader.

Related to #7013?

the latest main branch already fixed this, via upgrading to pytorch 2.4


[SPARK-41725][CONNECT] Eager Execution of DF.sql()

What changes were proposed in this pull request?
This patch allows for eager execution of SQL statements using the Spark Connect Data Frame API. When spark.sql is called it will send the query to the server where a DataFrame is created and cached with the Session. If the query is a command this transformation will immediately kick of the execution.
Why are the changes needed?
Compatibility
Does this PR introduce any user-facing change?
No
How was this patch tested?
UT

do we still need message SQL in relations.proto?

Yes, during eager execution the server generates the SQL relation if it's not a command.

merged into master/branch-3.4

I made a PR.

https://github.com/apache/spark/pull/40246


[SPARK-28831][DOC][SQL]Document CLEAR CACHE statement in SQL Reference

What changes were proposed in this pull request?
Document CLEAR CACHE statement in SQL Reference
Why are the changes needed?
To complete SQL Reference
Does this PR introduce any user-facing change?
No
How was this patch tested?
Manually tested newly added doc.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14557/
Test PASSed.

Test build #109500 has started for PR 25541 at commit 1a0b3b4.

Test build #109500 has finished for PR 25541 at commit 1a0b3b4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109500/
Test PASSed.

@huaxingao Could you please attach a screen-shot of the page ?

Test build #109553 has started for PR 25541 at commit 1f820a4.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/14610/
Test PASSed.

Test build #109553 has finished for PR 25541 at commit 1f820a4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/109553/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15358/
Test PASSed.

Test build #110371 has started for PR 25541 at commit 1594d52.

Test build #110372 has started for PR 25541 at commit e328b2e.

LGTM
@gatorsmile one question here.. so for this case, examples don't add much value. Its still good to have for completeness, right ?

Test build #110371 has finished for PR 25541 at commit 1594d52.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110371/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/15359/
Test PASSed.

Test build #110372 has finished for PR 25541 at commit e328b2e.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/110372/
Test PASSed.

LGTM
Thanks! Merged to master.

Thanks @dilipbiswal @gatorsmile @srowen


Resolve cyclic dependencies in in-place metadata initialization of value types

The central thrust of this PR is to get these metadata initializations off of swift_once and onto the metadata-request system where we can properly detect and resolve dependencies.  We do this by first introducing runtime support for resolving metadata requests for "in-place" initializations and then teaching IRGen to actually generate code to use them.
The remaining cases that are still using swift_once resolution of metadata initialization are:

non-generic classes that can't statically fill their superclass or
have resilient internal layout

foreign type metadata

Classes require more work because I'd like to switch at least the resilient-superclass case over to using a pattern much more like what we do with generic class instantiation.  That is, I'd like in-place initialization to be reserved for classes that actually don't need relocation.  Maybe that doesn't work given some of our goals of allowing system classes to be updated to become Swift in the future.
Foreign metadata should also be updated to the request/dependency scheme before we declare ABI stability.  I'm not sure why foreign metadata would ever require a type to be resolved, but let's assume it's possible.

@swift-ci Please test.

Build failed
Swift Test Linux Platform
Git Sha - cb936cd937969cf5c8c43edf675297cbcae2c085

Build failed
Swift Test OS X Platform
Git Sha - cb936cd937969cf5c8c43edf675297cbcae2c085

Build failed
Swift Test Linux Platform
Git Sha - cb936cd937969cf5c8c43edf675297cbcae2c085

@swift-ci Please test.

Build failed
Swift Test OS X Platform
Git Sha - ac14b779a7036cb517d99fd99db8b7937b8a5828

Build failed
Swift Test Linux Platform
Git Sha - ac14b779a7036cb517d99fd99db8b7937b8a5828

Failed to protect against protected in a Linux test; otherwise that build was clean.  Just doing a smoke test this time.

@swift-ci Please smoke test.


Added Webextension-Toolbox

Small cli toolbox for creating cross-browser WebExtensions.

I think it does too much, it even suggests including react as part of it. It overlaps with webpack, web-ext and babel. Too broad


Top level io ports end up unconnected

strangely, It's only the BIBUFs.
stuff like this:
          (net (rename id14732 "B2.PAD[13]") (joined
            (portRef PAD (instanceRef id14639))
          ))

wot I believe should look like this:
          (net (rename id14732 "B2.PAD[13]") (joined
            (portRef PAD (instanceRef id14639))
            (portRef (member ram_dataX 13))
          ))

The whole setup is here:
https://github.com/klammerj/y40k
(I've uploaded the IDS logfile, too)

I have now changed the verilog file to get rid of the inouts in the submodule to see if things change.
They did not.
Then hand edited the edf to wire up the PADs.
I am left with error messages about unconnected input ports.
It seems to be the AN2s from the AIG pass.
Looking at one of them the A input seems unconnected...
And a single unconnected mcu_addr[0] (this time an internal net is dangling)...

Fixed problem described in the OP in commit db7314b.
I simply do not understand what you are talking about in your second post. Please provide a test case (minimal Yosys Script + Verilog Input) and describe in what way the expected behavior differs from the actual behavior, if the problem is still present with current Yosys git head.


Enum doesn't work now

See https://github.com/haxetink/tink_typecrawler/commit/8743ffcc5f77a3d6fc7e5884552a4efd73815696#commitcomment-19657970

Edit: Please ignore point 2 in my comment.
And I have prepared a PR to fix point 1. #8


UI bug when clicking column of data table with one row

When I click a column of a data table that has only one row, more rows are "temporarily added" to the UI:
Before clicking the column

After clicking the column

We need to expand the iframe, but we can add extra space instead of table rows. Fixed in 2a7d598a593f4671819b625cac43066b57da8648.


VC_NAME_MISMATCH error upon adding second vaccination certificate

Technical details

Device name: Moto g6 plus
Android version: 9
App version:  2.4.3

Describe the bug
When trying to add the second vaccination certificate the app shows a VC_NAME_MISMATCH error. Both documents containing the QR code show the same name. The only difference I can see is that the date of birth is formatted differently: YYYY-MM-DD vs DD.MM.YYYY. It is also possible that this is related to handling of a Dr. title as part of the name, but at least on the certificates both are written identically.  The same certificates are accepted without problem by the CovPass App. I could work around this by deleting the first certificate from the app and just adding the second one but the behavior is still strange.
Steps to reproduce the issue

Add first vaccination certificate (It does not matter for the occurrence of the bug if it is from the first or second vaccination)
Add the second vaccination certificate
See error

Expected behavior
Both certificates should be added without error, as happens in the CovPass App.
Possible Fix
Looking at the printed data on the sheets I would expect it to be related to either the different date format or some minor difference in how the Dr. title is written.

A response here could likely also answer #3547

Huh, no idea how I missed that report. Name capitalization is the same for both certificates in my case.

@raeior Thanks for reporting.
Workaround: Delete all vaccination certificates and just scan the second one. IMMUNTY should be displayed  after 14 days of the second vaccination.

Corona-Warn-App Open Source Team

@raeior
In the announcement for version 2.13 of the Corona-Warn-App it describes an enhancement which might have solved your issue:
"In addition, the project team has improved the algorithm when assigning certificates to people. The Corona-Warn-App now assigns certificates with spelling errors in the name to the same person.
Previously, errors such as a different number of spaces between a person's first and last name caused the Corona-Warn-App to assign these certificates to two different people. This changes with version 2.13. For this reason, a certificate may no longer be displayed in the overview under the "Certificates" tab. However, the certificate has not disappeared, but is then displayed in the certificates section under the name of the corresponding person. Users can access it by tapping on their name under "Certificates"."

The error code VC_NAME_MISMATCH no longer exists in current versions of the app, so it would be impossible for it to appear.
This is from the old 2.4 version:
https://github.com/corona-warn-app/cwa-app-android/blob/25403e0e90c2ca1534a198a84380afdbb65c7672/Corona-Warn-App/src/main/java/de/rki/coronawarnapp/covidcertificate/exception/InvalidHealthCertificateException.kt#L31
Since the error can no longer occur, this issue should be closed.


Integration with AWS EKS and JFrog

Will this portieris works with AWS EKS? We are using JFrog to store the docker image and self-hosted notary server to sign images.

This should work, if you run into problems you can raise them here. Please re-open with any specific issues.