Consider parking_lot's RwLock instead of concread

Currently, read/write locks are only held for a very short amount of time. I originally choose concread because it supports concurrent writers/readers (so reads always remain unblocked), but I'm not sure it actually helps our specific benchmarks. One example is when a capsule is not initialized, you will need to grab a write, which is blocking.
The specific issue I'm facing with concread is that I can't support a Container::read_ref api with it right now. I'd need the ability to commit a write txn, and in that process, downgrade it to a read txn. I may file an issue over at concread to see if they'd support this, but if parking_lot's RwLock around a standard HashMap is faster anyways, I'm not sure it'd be worth it, as they support downgradable writes.

Based on the count benchmark in /examples (which isn't scientific by any stretch of the imagination), I got the following results. Perfect? No. Good enough to show trends and make architectural decisions? Probably, yes.

As you may predict, concread supports a large number of readers better. However, the RwLock with HashMap variant is faster for a smaller number of readers, and is always faster for writers. So the proper choice here is likely to depend upon the expected workload.
I'll sleep on this and see what I think later.

I am originally inclined to say that I am tempted to ditch concread, because:

This more easily opens the doors for no_std
Concread reads per second are significantly higher, but does this actually matter for a typical workload? I have a feeling most uses of ReArch may be on a server where you're interacting over some form of I/O, which I'd have to guess under most situations wouldn't be hitting that many reads per second anyways.
The writes per second can definitely be hit on a target server workload. Thus, if we can squeeze more writes out here

Also, if we do go down the no concread route, we will want to add in the container.read_ref method

Going to remove concread. Need to still add the Container::read_ref api.


Timestampable and Commands

I added the following field to my User entity I use with FOSUserBundle:
  /**
   * @ORM\Column(type="datetime")
   * @Gedmo\Timestampable(on="create")
   */
  private $createdAt;

It works fine when creating a user from a service. It fails from the command line.
php bin/console fos:user:create admin<EMAIL_ADDRESS>admin --super-admin

[Doctrine\DBAL\Driver\PDOException]
SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'created_at' cannot be null

maybe you use different environment to run command. it seems like listener is not injected to the entity manager. this issue is not related to extensions, but to a framework configuration or bundles you use. you should debug that yourself, the library works fine even in console


User profile testing

Create a user profile and edit it.
Create a dummy user profile and assign this dummy user profile admin privileges. Finally, delete the dummy profile.
(For the second part, you will need admin privileges. Let us know, what is the profile you are using and we will assign you the privileges.)

@bajiat : I am currently working on issue #877, Settings and localisation testing, I'll continue working on this issue#891 soon.

@bajiat , @brylie  @jykae :  Signing up on nightly.apinf.io is giving error "internal server error". Could you please check with this issue.
Also, please let me know if the latest version deployed, so that I can proceed working on this task.
Screenshot:

I got the same error, when trying to sign up / register as a new user.

Yes, this is probably related to a bug we are experiencing since the Umbrella 0.11 upgrade. We need to follow up with @GUI on the API Umbrella tracker to determine whether the problem may be with our integration package or with the API Umbrella Admin API.

Defect #893 created.

Nightly settings fixed, verify that registration works now, and close if successful.

@jykae , Sign up /register a new account on nightly.apinf.io was successful.
This is my user id: newtest_api1 .

@bajiat , I have signed up on nightly.apinf.io, could you please give me admin rights. Here is my username: newtest_api1

@jykae , Thanks for providing profile admin privileges.

This task has been completed and please find the below issues reported:
https://github.com/apinf/api-umbrella-dashboard/issues/902
https://github.com/apinf/api-umbrella-dashboard/issues/904
https://github.com/apinf/api-umbrella-dashboard/issues/909
https://github.com/apinf/api-umbrella-dashboard/issues/903


Two Cassini mission phases have no data

The Cassini Mission Phases "Equinox Mission" and "Solstice Mission" have zero search results. They have probably been superceded by things like "Extended Mission" and should be removed from the available options.

Firstly, the problem statement is not true.  See attached screenshot.  It seems that every value of Cassini Mission Phase has an instance somewhere in the holdings.

Secondly, the Cassini LDD has multiple synonymous values for mission_phase_name.  I strongly disagree with this design decision, but Atmospheres Node are the stewards of this LDD and it was their decision.  Basically, different instrument teams used different names for the same thing, and they decided to change the standards to fit the data rather than the other way around.  For example, "Equinox Mission" and "Extended Mission" are synonymous with each other, as are "Solstice Mission" and "Extended-Extended Mission."
It would be best (but is not urgent) for us to eliminate synonymous values of this attribute.  For example, I would prefer we use the ones you mention and remove the ones with "Extended" in the name.

I'm realizing that my response here is adding some PDS4 concepts to the conversation, while you are talking about a strictly PDS3 situation.  Nevertheless, I think my basic points remain valid.

Sorry, this was user error. I was looking only at Cassini ISS, which inadvertently revealed the exact issue you brought up.
Note this has nothing to do with the LDD, since this is a PDS3 data set.
Perhaps when we convert to importing the PDS4 version we can map the mission phase names as you suggest.

Yes, this is PDS3.  However, it's interesting to note that the LDD has been set up in such a way that this problem will be perpetuated in PDS4.
Yes, we could do that.  In fact, nothing prevents us from going ahead now with a mapping as part of the OPUS import.  However, as I said, it's not a priority.


more 3.0.7 features

I was intending to include this with 3.0.7

Fixes the default sourcetype issue and cloud_details grains issue. 👍


Not possible to include CSS & Javascript

Hello,
my app fails when I include both CSS and Javascript, only one of them is loaded
Shiny version: 0.10.1
Example:
# app.py
from shiny import ui

ui.page_fluid(
    ui.head_content(ui.include_css("custom.css"), ui.include_js("script.js")),

    # You can also inline css by passing a dictionary with a `style` element.
    ui.div(
        {"style": "font-weight: bold;"},
        ui.p("Some text!"),
    )
)

What do I expect? That both custom.css and script.js will be added to the HTML.
What happens? One of them generates a 404 error

Workaround: Create a separate folder for each file. Problem should be somewhere here: https://github.com/posit-dev/py-shiny/blob/33c8c79b7edf28f2e0d1d1e2b6051eed084d15ad/shiny/ui/_include_helpers.py#L215

Furthermore, I discovered that when you use (in the core syntax)
app_ui = ui.page_output("page_generator")

and return both in the headers, it will not be copied too
    @output
    @render.ui
    def page_generator():
        return ui.page_fillable(
            ui.head_content(ui.include_css(Path(__file__).parent / "css/style.css")),
            ui.head_content(
                ui.include_js(Path(__file__).parent / "js/script.js"),
            ),


Snyk-related dependency upgrades

Information

Fixes https://jira.cms.gov/browse/QPPFC-1468

Changes proposed in this PR:

Updating versions of: spring-boot, spring-security-jwt, jackson, guava, aws-java-sdk, aws-dynamodb-encryption-java
qpp-data-restoration now uses dependency versions defined in the root pom.xml

Notes

This should reduce the findings from Snyk (high/med/low) from 41/17/1 to 24/6/0
The remaining findings relate to:

A not-yet patched vulnerability in jackson
An old version of jackson used by aws-java-sdk
Outdated versions of guava used by reflections and jimfs

Checklist

[x] All JUnit tests pass (mvn clean verify).
[n/a] New unit tests written to cover new functionality.
[n/a] Added and updated JavaDocs for non-test classes and methods.
[x] No local design debt. Do you feel that something is "ugly" after your changes?
[n/a] Updated documentation (README.md, etc.) depending if the changes require it.

Handled manually in newer pr.


p5.js (ml5) snippet

Hello! I am working on an example for Teachable Machine and p5.js. I made some adjustments to the current snippet in the beta. My suggested code is at the bottom. I've added some comments and also removed the need for DOM elements opting to render the label text to the canvas itself.

One small issue is that the video is not mirrored, I am thinking about how to handle this.
Also, I think rather than copy/paste it might make sense for the interface to provide a link to a p5.js web editor template where the user just has to copy in the model URL. At a minimum, something should note that the code will only run with the p5 and ml5 libraries, e.g.:
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.9.0/p5.js"></script>
<script<EMAIL_ADDRESS>
Would you like me to create the template as part of the ml5 web editor account / examples?
(I should note 0.4.1 does not yet exist, but will include a bug fix that makes the example run, see: https://github.com/ml5js/ml5-library/pull/650)
// Classifier Variable
let classifier;
// Model URL
let imageModel = 'https://storage.googleapis.com/tm-models/WfgKPytY/model.json';

// Video
let video;

// To store the classification
let label = "";

// Load the model first
function preload() {
  classifier = ml5.imageClassifier(imageModel);
}

function setup() {
  createCanvas(320, 260);
  // Create the video
  video = createCapture(VIDEO);
  video.size(320, 240);
  video.hide();

  // Start classifying
  classifyVideo();
}

function draw() {
  background(0);
  // Draw the video
  image(video, 0, 0);

  // Draw the label
  fill(255);
  textSize(16);
  textAlign(CENTER);
  text(label, width / 2, height - 4);
}

// Get a prediction for the current video frame
function classifyVideo() {
  classifier.classify(video, gotResult);
}

// When we get a result
function gotResult(error, results) {
  // If there is an error
  if (error) {
    console.error(error);
    return;
  }
  // The results are in an array ordered by confidence.
  console.log(results[0]);
  label = results[0].label;
  // Classifiy again!
  classifyVideo();
}

cc @joeyklee

@HalfdanJ Here is a proposed snippet for sound classification. Same requirements (ml5 0.4.1+ and p5.js).

// Global variable to store the classifier
let classifier;

// Label
let label = 'listening...';

// Teachable Machine model URL:
let soundModel = '________';

function preload() {
  // Load the model
  classifier = ml5.soundClassifier(soundModel);
}

function setup() {
  createCanvas(320, 240);
  // Start classifying
  // The sound model will continuously listen to the microphone
  classifier.classify(gotResult);
}

function draw() {
  background(0);
  // Draw the label in the canvas
  fill(255);
  textSize(32);
  textAlign(CENTER, CENTER);
  text(label, width / 2, height / 2);
}

// The model recognizing a sound will trigger this event
function gotResult(error, results) {
  if (error) {
    console.error(error);
    return;
  }
  // The results are in an array ordered by confidence.
  // console.log(results[0]);
  label = results[0].label;
}

Awesome, thanks! Ill copy these in. We will soon move it over so the snippets live in this repo.
And regarding the link to p5.js, I think that would be great! What would that link look like exactly?

Closing this since we just merged the PR for this.

Solved by: https://github.com/googlecreativelab/teachablemachine-community/pull/21


Preparing for release 3.7.2

Update version
Update CHANGELOG

Pull Request Test Coverage Report for Build 234

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 233:
0.0%

Covered Lines:
400

Relevant Lines:
400

💛  - Coveralls


plan-bench Requirements no links

The Requirements in the README of plan-bench about Fast Downward, VAL and PR2Plan related links jump to empty pages and cannot be downloaded.
In addition, the CONFIG.Yaml file does not exist in the /plan-bench/config/ file

hi @LWMin, have you downloaded the files? I am also having this issue

I have updated the links to download the required tools. I have also provided some tools in the repo itself. Hope that is useful. Thank you.


It can't support sprite collision?

It can't support sprite collision?

@freezyh No, just rectangular bounding boxes and circles.

@freezyh hi！How did you solve it? Thanks


docs: project names, API calls, custom controlled vocab, extensions

project name is generic rather than month based; closes #24
identifiers/resource_types updated in docs
refactor customization/configuration in one file
document custom vocabulary; closes #42
document: metadata extensions; closes #38

@fenekku could you update docs/install/index.md lines 60 and 64. They mention version of the CLI to be 0.10.1. Alternatively we can remove it, I think pip works well to install latest and people should look in pypi. WDYT?


[catr] tests unreliable?

Hey there,
I'm following along the examples in the book and I can't seem to get the tests for catr to pass...

When I use the debugger to get some idea it seems that the test doesn't read the expected outputs correctly:

But if I do a hex-dump of the file contents it seems to be just fine?

I tried re-generating the outputs with the script in this repository but that doens't seem to help.
Anyone have an idea?
The code is from the tests itself, unmodified:
fn run(args: &[&str], expected_file: &str) -> TestResult {
    let expected = fs::read_to_string(expected_file)?;
    Command::cargo_bin(PRG)?
        .args(args)
        .assert()
        .success()
        .stdout(expected);
    Ok(())
}

Hey @florianhumblot , I had exactly the same issue and the test output is super confusing. What fixed it for me was adding final line breaks to all the input files in my editor and regenerating the expected directory:

You're welcome to copy my expected directory and try it out https://github.com/richo225/command-line-rust/tree/d586c7afbd4654e52b065da6e3185fe54a4274b2/catr/tests/expected

@richo225 thanks for the hint! I already added line breaks to the end of file, but changing the line endings to LF rather than CRLF seems to have fixed it. Kind of a weird issue to have lol


Improved feedback

Moves feedback state/storage from browser local storage to disk.
Adds a third state (minimized) for when the user has hidden the Wnat a free t-shirt button but has not submitted the form. A t-shirt icon is displayed in place of the text button in this case.

Resolves #723

I am unsure why local storage was not behaving correctly in colab, and debugging it was proving to be cumbersome. So I opted to move the state out of the browser.

Pardon the delay. Comments addressed.

Pardon the delay. Comments addressed.


Connecting LDR

Thanks for the great addition, can you please confirm how to connect the LDR to Pin35 for your code to operate?
Is it as per this ESP Trinity schematic?

https://github.com/witnessmenow/ESP32-Trinity/blob/master/hardware/ESP32Trinity/V1.2/Schematic_ESP32-Trinity_2021-08-27.png

Hi Anthony. Sorry, it's exactly this. I'm going to update the documentation.

If you get the Trinity, is it a matter of just soldering the LDR on the provisioned pads?

Yes just solder one to the marked position.

Awesome software, would be great to be able to select GPIO used for LDR in settings page to avoid users to recompile.
I'm Author of an old OpenSource shield for ESP32 D1 Mini and as I wanted to keep compatibility with ESP8266 D1 Mini, LDR is connected to A0 AKA GPIO36 on ESP32 D1 Mini

@anthonyjclarke I added the LDR wiring in the README
@hallard Thanks for the feedback, I will put this change in the backlog. If you don't mind, I will link your shield as one of the options to run the Clockwise.

Awesome software, would be great to be able to select GPIO used for LDR in settings page to avoid users to recompile.
I'm Author of an old OpenSource shield for ESP32 D1 Mini and as I wanted to keep compatibility with ESP8266 D1 Mini, LDR is connected to A0 AKA GPIO36 on ESP32 D1 Mini

+1 to this. I have some of Charles great boards as well as the I2S board from @witnessmenow boards and would like this flexibility

I agree. I will include the option to configure the GPIO in the next release.

I agree. I will include the option to configure the GPIO in the next release.

Awesome, thanks 🙏

If you don't mind, I will link your shield as one of the options to run the Clockwise.

Of course you’re welcome


Docker compose watch command fails

When I run the command docker compose watch as suggested in the docker documentation here: https://docs.docker.com/guides/getting-started/develop-with-containers/
I get the following results

This seems to be to be a root cert issue that should be indigenously solved inside the container?

@archaica Use  docker compose up --watch instead.
Will get the guides fixed.

That yields the same error.

@archaica I suspect this is due to network connectivity problems or interference from a proxy server that's corrupting the certificate chain. Can you try it once on your laptop outside your company network?
Try temporarily disabling firewalls or proxies to see if the issue persists. Access the registry server from a different network to rule out network-specific problems.
Also, what happens when you run npm install command locally for each of backend/ and client/ directories. Does it work well?

PS C:\Users\DELL\getting-started-todo-app> docker compose up --watch
[+] Running 3/3
✘ proxy Error      context canceled                                                1.7s
✘ phpmyadmin Error context canceled                                                1.7s
✘ mysql Error      Head "https://registry-1.docker.io/v2/library/m...              1.7s
Error response from daemon: Head "https://registry-1.docker.io/v2/library/mysql/manifests/8.0": unauthorized: incorrect username or password

getting this error

@harigithubac The error message you're seeing indicates that Docker Compose is encountering issues pulling images from Docker Hub due to invalid credentials. Did you make any changes related to the database credentials? Also, check if your system is not behind the firewall.

@harigithubac The error message you're seeing indicates that Docker Compose is encountering issues pulling images from Docker Hub due to invalid credentials. This is unexpected, as the images used in the compose file typically don't require login to Docker Hub. Are you using a corporate laptop? If so, ensure you're not restricted by the company's firewall.

No using personal lap

That yields the same error.

that must be different error if your running on windows port 80 is reserved change the port in compose.yaml
  proxy:     image: traefik:v2.11     command: --providers.docker     ports:       - 81:80     volumes:       - /var/run/docker.sock:/var/run/docker.sock


Edit some channels

This PR is for edit new channels.

You seem to be getting confused. That PR I did days before what you say, which by mistake was merge to branch. Better, I will fix.


Nextjs config won't export when using Nextra - next.config.js

Hi,
I am getting issue that i can not wrap my head around.
I am trying to export nextra nextJs with docs theme. But as from version 13.3 is the next export deprecated I am using the next config.
Here i am encountering an issuse where i either export:
const withNextra = require('nextra')({
  //Nextra
  theme: 'nextra-theme-docs',
  themeConfig: './theme.config.tsx',
  unstable_staticImage: false,
  output:"export"
});

module.exports = withNextra()
;

or:
/**
 * @type {import('next').NextConfig}
 */
const nextConfig = {
  output: 'export',

}

module.exports = nextConfig

I have tried to export both:
module.exports ={

  ...withNextra(),
  ...nextConfig

}

which gives me a rewrites error.
Steps to Reproduce

https://nextra.site/docs/docs-theme/start - follow this example, and try to export it

https://nextjs.org/docs/pages/building-your-application/deploying/static-exports

#Expected Behavior
Expected HTML , CSS  and JS outut
Real Behavior
Either when using the nextra config - it is exported to be used with next start
Either if switching to the nextJS export config - the output is missing all static pages and only _app and 404.html gets generated.
Thanks for help

how did you solve that?


Build for electron

I added electron to my project, and i'm trying to get npm run build to create a bundle for it that doesn't use the service worker or hash the filename
I created an additional (fixed) bundle named 'desktop' in the config and tried adding it to the webpackConfigFactory, but it won't compile when i run the command. Instead I get this error:

ERROR in ./src/desktop/index.js
Module parse failed: src/desktop/index.js Unexpected token (35:6)
You may need an appropriate loader to handle this file type.

It seems that it's falling down on the JSX. What am I missing to make this work? Do I need to edit another config file?
Any suggestions much appreciated :)

Hey @eurobob
Hmmm, by the sounds of it you need to make sure the "babel-preset-react" is getting added to the "isNode" builds.  Check the project configuration and update the babel configuration appropriately.  Most like you just need to remove the if-statement.

would that be in the config/index.js file?

Yep. :)

Isn't that already happening on line 335?

Probably off topic.
Webpack support special targets for electron builds.
I don't know what are the consequences on bundles built.
But I think that it's an useful information which need to be shared with you sirs.

For a bit more context, I pointed the electron compilation at the client build and it worked. It's just a pain because of the hash. All i'm looking to do is put the same client build in a desktop/electron folder with no hash and no service worker. How would I go about doing this?

How would I go about doing this?

Remove hash in file names included in the bundle?
Disable the service worker?
The both?

Both. What I want is 3 folders in my build output:
• Client
• Server
• Desktop
I don't want to alter the client build. I want that to compile as it currently does. I just want to compile exactly the same thing into the desktop folder with the service worker disabled (which I believe just requires removing it in the src index for desktop) and not to hash it.
I'm having trouble though. Since it's not a node bundle it doesn't belong in the additional bundles section, so i'm having to hack the web pack config a bit, but it doesn't recognise the JSX

@eurobob

i'm having to hack the web pack config a bit

Yes, it's the right way. You need to create logic about your desktop target (isDesktop and ifDesktop constants must be created and used to disable hash in file names and disable the service worker).
Share a gist of your webpack factory if you encounter difficulties.
@ctrlplusb
This topic show limitations of the current webpack factory. If you plan to adjust the webpack factory, please takes this case in consideration my friend.

Open to PRs, but honestly I like the web focus.


Improve browserify support

Fixes #230. Supersedes #231.

Thanks! I plan to make release after #228 fixed or in a week.

Would you be willing to release this change as a patch version, and release #228 afterward as a feature/minor version?

I'd prefer to increase minor version for this change too.

Fair enough.

Any chance this could make it to npm?

That does not work well, because replace module with {} instead if ignoring. It became a problem, when you do really need to use esprima.


Thread – Join button reappears in header when try to join thread after leave it

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Version Number:  v9.0.10-2
Reproducible in staging?: yes
Reproducible in production?: no, new feature
If this was caught during regression testing, add the test name, ID and link from TestRail: https://expensify.testrail.io/index.php?/tests/view/4747620
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Expensify/Expensify Issue URL:
Issue reported by: Applause Internal Team
Slack conversation:
Action Performed:

Go to https://staging.new.expensify.com/
Log in
Open chat with user B
Send a message
Right click and select Reply in thread
Send a comment in the thread
Click on header
Click on Leave thread
Click on Join thread button on the header

Expected Result:
Join button disappears, user joined to thread
Actual Result:
Join button reappears, user is not joined to thread
Workaround:
unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[x] Android: Native
[x] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[ ] MacOS: Desktop

Screenshots/Videos
https://github.com/user-attachments/assets/d471df36-419e-48e7-b30e-fc226f6bbbf2

View all open jobs on GitHub

@bernhardoj @akinwale Seems like this might be related to https://github.com/Expensify/App/pull/45136 maybe we should revert the PR  and cover these newly found edge cases

Confirmed that it comes from https://github.com/Expensify/App/pull/45370/ as suggested here by reverting it locally.
/cc  @dominictb @thesahindia if you can think of a fix. In the meantime I'll prepare a revert CP just in case.

No payment is needed as this was internal and a C+ was not assigned due to comment here. Closing!


Creates go.gitignore

Go gitignore for use with deb and gin
The gin live-reloader for Go web applications https://github.com/codegangsta/gin creates a gin-bin file for serving the live server. This file is a binary file and is updated after each start of the gin server but not deleted when stop. To avoid checking it in i added the gin-bin to the ignore list.
The vendor folder is create by the deb dependency manager of go. The versions and dependencies are stored in the Gopkg.lock and Gopkg.yml and can be generated by deb ensure on the build system. So there is no reason checking the vendor folder in.
https://github.com/codegangsta/gin
https://github.com/golang/dep

@strati1991 we already have a Go.gitignore template in this repository, and I'm not clear on how this differs from that.
Are we able to have a unified template?

Closing this out due to inactivity.
@strati1991 if you're still interested in submitting this contribution, please open a fresh pull request and address the outstanding questions from this review.


error fixed at chapter 6 /7.md

changed in chapter-6/7.md as seen. where we can observe in the Unigram session. where P is 17. but it was mentioned as 5 in the formula.
https://huggingface.co/learn/nlp-course/chapter6/7?fw=pt . the following link is for ref.

@HuggingFaceDocBuilderDev

@lewtun


Update k8s related pkg and use k8s cri client as e2e test client

[x] Update k8s related pkg
[x] use k8s cri client as e2e test client

Signed-off-by: Xianglin Gao<EMAIL_ADDRESS>

This change is

@feiskyer ready to be reviewed

@feiskyer update vendor and make it be even with k8s.

LGTM. Thanks.


Refactor validations under validate command

Motivation
Root was getting unwieldy
Additional Notes

Added service check validation to ci
Got rid of manifest set for now, we'll add back as separate command in future if there is any desire

Codecov Report

Merging #2593 into master will decrease coverage by 6.67%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #2593      +/-   ##
==========================================
- Coverage   83.88%   77.21%   -6.68%
==========================================
  Files         644       21     -623
  Lines       36411      610   -35801
  Branches     4423       65    -4358
==========================================
- Hits        30543      471   -30072
+ Misses       4554      113    -4441
+ Partials     1314       26    -1288

@gzussa We can't because that would produce a name conflict with validate.py.

Could we find another name like validations?

We already do this though


try https://www.youtube.com/watch?v=nTGWvvUTSeQ&t=502s

https://www.youtube.com/watch?v=nTGWvvUTSeQ&t=502s


/j

I understand /j enables unbuffered IO, but this doesn't seem to be documented on this page.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bf2281d4-1470-0658-ec5a-687fe2cd451c
Version Independent ID: 2ce2be3e-e3a2-790c-3bff-90efe7318450
Content: robocopy
Content Source: WindowsServerDocs/administration/windows-commands/robocopy.md
Product: windows-server
Technology: manage-windows-commands
GitHub Login: @coreyp-at-msft
Microsoft Alias: coreyp

Hi MS, should really fix this one

https://ss64.com/nt/robocopy.html

Thanks for the motivational comments and the useful reference. I may just make a PR to update that page, but I have to postpone it for just a few days, until this main section has finished receiving a makeover or overhaul for other types of changes.

Thanks for the feedback. This article now includes the /j parameter.
#please-close


Don't assert on union/isect flatten

It's possible to hit the flatten pass on types that have already
been flattened, for example when generating an anonymous type for
a lambda. Instead of asserting that the child count is 2, the
flatten step can be skipped if the child count is greater thant 2.

Please add minimal test case for this as a regression


Nature's Grace

Druid talent Nature's Grace would be a good addition.
Talent https://www.wowhead.com/wotlk/spell=61346/natures-grace
Buff https://www.wowhead.com/wotlk/spell=16886/natures-grace
It's a bit atypical because it gives a buff for 3 seconds, rather than an effect "on next cast". However, due to the short duration, the buff will most likely help one spell cast (or two at most) so it makes sense to treat it as a "next cast" ability.
The chosen location would be "Top", which is not used by balance druids yet, with the serendipity texture because "the kind of moon-flashy-proc-thingy kinda fits the balance aspect", as mentioned here.

Implemented in #101.


[ArchBoard] Review the forceCloseHandles API

This API is interesting as it uses paged responses but is closer to a long running operation based on the service implementation. The API should be reviewed to be converted to a specialized LRO and potentially having different API methods for a single handle closure (won't be LRO) and a many handle closue (may be LRO).

Based on ArchBoard discussions there will be two APIs for this REST endpoint:

forceCloseHanlde(String handleId)
forceCloseHandles(boolean? recursive)

@conniey could you help load balance this issue?

Sounds good.


Duplicate calls of APPlatformController interface for the same instance

There is a race condition for APP timer initialization, which results duplicate calls of APPlatformController interface for the same instance.
in APPTimerService.java
public void initTimers() {
    Collection<?> timers = timerService.getTimers();
    if (timers.isEmpty()) {
        try {
            String timerIntervalSetting = configService
                    .getProxyConfigurationSetting(PlatformConfigurationKey.APP_TIMER_INTERVAL);
            long interval = Long.parseLong(timerIntervalSetting);
            timerService.createTimer(0, interval, APP_TIMER_INFO);
        } catch (ConfigurationException e) {
            timerService.createTimer(0, DEFAULT_TIMER_INTERVAL,
                    APP_TIMER_INFO);
            logger.info("Timer interval not set, switch to default 15 sec.");
        }
    }
    logger.info("Timer initialization finished");
}

The check "timers.isEmpty()" is not synchronized nor locked, so there is a chance for multiple timers to be create if two or more calls of initTimers() come in a very short time.

The timers in Java EE are transactional. But the access to the the timers is not serialized. Each stateless bean can access the same timer in parallel.  As a result, one transaction cannot see the timers from the other parallel transaction. At the end, both transactions are commited and 2 timers are created, even if the intention of the application logig is to create one.
The problem is fixed with using a Singelton bean (APPTimerServiceBean) with bean managed concurrency to avoid this situation.


Want to PHP trim() body, query parameters, and arguments

I'd like to trim all incoming data, whether it be in the body, a query parameter, or argument (route placeholder).
A middleware seems to be the best place to do this work. I searched online but couldn't find any existing code, so I wrote this:
// Middleware to sanitize (trim) all incoming data
$app->add(function (Request $request, Response $response, callable $next) {

    // Trim any data in the body
    $body = $request->getParsedBody();
    // Make sure the body has some data before we try to trim it
    if (is_null($body) === false) {
        // Pretty basic sanitizing - right now just trimming the values
        $body = filter_var($body, FILTER_CALLBACK, ['options' => 'trim']);

        // Replace the posted body with the cleaned copy
        $request = $request->withParsedBody($body);
    }

    // Trim any data in the query parameters
    $query_params = $request->getQueryParams();
    // Make sure the query params have some data before we try to trim them
    if (is_null($query_params) === false) {
        // Pretty basic sanitizing - right now just trimming the values
        $query_params = filter_var($query_params, FILTER_CALLBACK, ['options' => 'trim']);

        // Replace the posted query parameters with the cleaned copy
        $request = $request->withQueryParams($query_params);
    }

    // And finally, trim the arguments, if any
    $route = $request->getAttribute('route');
    $arguments = $route->getArguments();
    if (is_null($arguments) === false) {
        // Pretty basic sanitizing - right now just trimming the values
        $arguments = filter_var($arguments, FILTER_CALLBACK, ['options' => 'trim']);

        // Replace the arguments with the cleaned copy
        $route->setArguments($arguments);
    }

    // Go on to the next callable
    $response = $next($request, $response);

    // We're all finished
    return $response;
});

Thoughts?

This topic should be discussed on the forum instead of in the issues section. Head over to https://discourse.slimframework.com. Thanks


Client timeout fails deployment even though all resources deploy successfully

We have an app with a fairly large container with a long startup time.  I can't get it to deploy fast enough to not hit a timeout.  All the resources still get installed in kubernetes but it always marks the helm deployment as FAILED with no way to fix it.
I would expect the client to upload the chart to tiller and tiller would still continue to insure the k8s resources were created and the deployment succeed even if the client lost connection.

This is a good proposal for Helm 3 in the form of refactoring tiller into a CRD controller. See #3089 for more info.


Add query execution time and query id to audit log query messages.

Scope & Purpose
Log query runtime and query id in audit log query messages.
This is a backport of the functionality in https://github.com/arangodb/arangodb/pull/13260, but without any of the refactoring.
Enterprise companion PR: https://github.com/arangodb/enterprise/pull/611

[ ] :hankey: Bugfix (requires CHANGELOG entry)
[x] :pizza: New feature (requires CHANGELOG entry, feature documentation and release notes)
[ ] :fire: Performance improvement
[ ] :hammer: Refactoring/simplification
[x] :book: CHANGELOG entry made

Backports:

[x] No backports required

Related Information

[x] Enterprise PR: https://github.com/arangodb/enterprise/pull/611

Testing & Verification

[x] This change is a trivial rework / code cleanup without any test coverage.
[x] The behavior in this PR was manually tested

Link to Jenkins PR run:
http://<IP_ADDRESS>:8080/view/PR/job/arangodb-matrix-pr/13304/

Tests blue

feature-3.6/audit-query-logging-improvements

Tests blue except known failure in time-dependent unreliable test.


Fix posting messages

pass prev message id when posting (needed to show errored messages). add flag to disable disconnect overlay. remove duplicate selectors
@keybase/react-hackers
@mmaxim when i do this i still don't get errored messages. i actually get a duplicate of my last valid message. let's discuss

I'll put this branch and try it out.

This seems to work with https://github.com/keybase/client/pull/5331


[WIP][HUDI-2706] refactor spark-sql to make consistent with DataFrame api

What is the purpose of the pull request

refactor CreateHoodieTableCommand;
use TBLPROPERTIES syntax to pass config instead of OPTIONS, and sync hudi's config to properties of the table rather other properties of the storage of the table in hive metastore, also keep compatible with OPTIONS syntax;
force to provide PrimaryKey, so that make Update/Delete available;
hudi spark-sql decouple from metastore as far as possible, get config from local hoodie.properties first;
modify operation and related configs when insert/merge, to make consistent with dataframe api.
add parameter validation for recordKey, preCombineKey, keyGenerator, even if those parameters are defined by different keys.

Brief change log
(for example:)

Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)

Added integration tests for end-to-end.
Added HoodieClientWriteTest to verify the change.
Manually verified the change by running a job locally.

Committer checklist

[ ] Has a corresponding JIRA in PR title & commit

[ ] Commit message is descriptive of the change

[ ] CI is green

[ ] Necessary doc changes done or have another open PR

[ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@hudi-bot run azure

The primary key syntax is consistent with other relational databases
create table test_hudi_demo (
    id int,
    name string,
    price double,
    ds date)
stored as hudi
primary key (id)
lifeCycle 300

@YannByron just did a rough pass over the changes. will do another round on details soon.

@hudi-bot run azure


After updating from 0.28.0 to 1.3.2, map doesn't re-center around new coordinates, only on Android

Summary
Not sure this is a bug but thought there was enough information that it might be.
I've been using version 0.28.0 for years and it works great. After updating to 1.3.2, when you give the map a new location, it doesn't re-center around the new coordinates until you interact with the UI (refer to the videos below to see what I mean).
I didn't see any changes to core functionality in the docs/readme and, as it works as expected on iOS, I'm hesitant to change anything in my code. While trying to debug I noticed that when you change the coordinates, onMapLayout is not called again. That was the only difference I could find between the old coordinates and new. But the same thing happens with version 0.28.0 so not really sure that matters. fitToCoordinates is called   in onMapLayout and when you input a new location.
Reproducible sample code
This is pretty tough to supply as my codebase is huge and intertwined. I get if you can't help without it but just thought I'd try in case anyone has any thoughts off the bat of why this might happen.

Steps to reproduce
The only thing that might be possible is using the test app. See if different coordinates cause the map to re-center with 0.28.0, then upgrade to 1.3.2 and try the same thing.
Expected result
Here you can see I start with the user in Hawaii and the different points to choose from on the East Coast and the map is zoomed out to accommodate everything. When I change the user to where the points are, the map correctly zooms in.
https://user-images.githubusercontent.com/81834841/214172268-c0a1e534-4d73-4574-b7b3-6a03d6b4dd46.mp4
Actual result
Here you can see I start with the user in Hawaii and the different points to choose from on the East Coast and the map is zoomed out to accommodate everything. When I change the user to where the points are, you can see that the user's location has indeed changed but the map stays zoomed out, as if it thinks the user is still in Hawaii, until I click on one of the cards representing one of the points.
https://user-images.githubusercontent.com/81834841/214172399-edfea75d-88a6-48f4-b1a3-1eb617495039.mp4
React Native Maps Version
1.3.2
What platforms are you seeing the problem on?
Android, iOS (Google Maps)
React Native Version
0.67.4
What version of Expo are you using?
Not using Expo
Device(s)
Android physical devices and emulators
Additional information
Side notes, you can also see the markers flashing a bit in version 1.3.2, the font for the word "Widener" is suddenly very bold, and the app started constantly refreshing as well. Any insight on why these might've changed is appreciated too. Thank you!

Some progress? Im trying to fix this problem since yesterday and updated my rn since was older than the specified (0.64.3) but the problem persist. My app is rerendiring and the marker as well, but the zoom is going into the ocean.

@Fersy1998 Not yet, will post here if I discover anything.

Hi @gabriellend, I can't find any onMapLayout method in 1.3.2 nor 0.28.0, did you mean the onLayout event? Also, it doesn't look like fitToCoordinates should trigger without using onLayout, check its reference in the MapView docs for 1.3.2. Might the misuse of onMapLayout instead of onLayout be why you encouter this issue in your project? If it is the case, you might also investigate why it is working in 0.28.0, you might rerender something where there is no use for it.
Such a big upgrade 0.28.0 -> 1.3.2 might introduce breaking changes so you'll probably have to scout for deprecated methods that you use in your code and change it accordingly if there are any. You can also check fitToCoordinates in 0.28.0 and in 1.3.2 for reference.

@polygbrl Thank you for your response! You are 100% right, my mistake onMapLayout is not a thing in the package. That was my own function that is called when onMapReady is called. So replace onMapReady anywhere I wrote onMapLayout. I do see what you mean by fitToCoordinates being recommended in onLayout, I will play around with that as well as look at the differences between both versions of fitToCoordinates.

So @gabriellend, onMapReady is a callback that "is called once the map is fully loaded" according to the docs. So I think something, probably your state, triggers a rerender of the map and therefore triggers as a side effect fitToCoordinates. So, you should check if something in the methods, events or callbacks you use on your map had changes that might explain why it is not re-triggered in 1.3.2. I hope this is leading you to the right direction!

@polygbrl I kind of forgot a big thing that I mentioned in my original post – it's working fine on iOS. It seems to me if that's the case, it's probably not something to do with my code. Do you agree, or think your comments may still apply and I might have to do something along the lines of If (config === "android")...?

@gabriellend I agree there shouldn't be any difference between iOS and Android, so equivalently, I'd consider there shouldn't be any difference in your method calling fitToCoordinates. Therefore the if (config === "android") part shouldn't handle that part of the logic performing the desired behavior.
I just tried the fitToCoordinates method in various situations both on an iOS physical device and an Android physical device & an Android emulator with 1.3.2 and I can't reproduce your issue.
While double checking your video, I can see that the fitToCoordinates method seems to be fired only when selecting a Meeting point (like in the not working example). In your working example, no Meeting point is selected and it looks to me that it is just zoomed into your location marker. Are you sure the effect you describe in your working example is indeed triggered by fitToCoordinates and not just a side effect of some rerendering that could have been confused with a fitToCoordinates result?
I'm sorry that I insist you double check your code but I don't have much to look into and therefore might also be wrong on how you handle your logic.

@polygbrl No worries, I totally appreciate you taking the time to help me! I'll will definitely look into it, might take me a week as I'm also working on some other things. Thanks for your help!

@polygbrl and anyone else that stumbles on this. I fixed the zoom issue by adding a call to fitToCoordinates to the onLayout method of the ScrollView which wraps the meeting point cards. I still am unsure why it worked on iOS but not on Android. The icons still blink more than I'd like/certain words are bolder than before on Android but I will continue updating to the latest react native/react version and see if that helps. Thank you for your help! I will close this now.

I've updated from version 0.31.1 to 1.4.0, all good on IOS but on Android the markers keep flickering and the map is overall very slow to render or when the user tries to interact with it.
At the moment I've removed the new google maps Renderer and everything works fine, will take a closer look when/if I have the chance but if someone needs a "quick" fix this might help or point in the right direction - not sure this is the best solution and probably it's not on the long run
here are the [docs] about it(https://github.com/react-native-maps/react-native-maps/blob/master/docs/installation.md#using-the-new-google-maps-renderer) section
here are some known behavioural changes Github


Fix version tracker issues and improve its capabilities

Fix some known issues of etl.helpers.VersionTracker and improve its capabilities:

Now an error is raised if an active dependency of a step is missing (even if it's not in the archive dag).
The warning that lists archivable steps shows all archivable steps possible (calculated recursively).
Errors and warnings are much more human-readable.
The "steps dataframe" now only has one row per unique step, and has other useful columns, including the list of direct usages/dependencies, the list of all usages/dependencies, the list of all step versions, the name of the dag file where the step is defined, and the path to its code.
There is a check now (that will run on every make test) to ensure all steps have code. It has happened a few times that we have removed old code (e.g. from fasttrack) but forgot to remove/archive the corresponding steps in the dag.
I've removed many steps from the archive dag that relied on walden. Such steps are not replicable (at least not easily), so we should little by little remove them.
I've put some order in the walden ingest folder, to have a similar structure to snapshots, and removed unnecessary code from that folder. I created some dummy files to avoid warnings when checking for the code of old steps.
I created a "snapshots_archive" folder, since we currently don't have a place where to dump archive snapshot scripts (as we do for data steps).

Also, unit tests are added for the new features above.
More context in https://github.com/owid/etl/issues/1821
There is no need to review the walden and archive dag files. The only files that need review are:

[x] etl/helpers.py
[x] etl/paths.py
[x] tests/conftest.py
[x] tests/test_helpers.py

Summary by CodeRabbit

Refactor

Updated and renamed functions related to step dependencies and usages in ETL processes for improved clarity and efficiency.

New Features

Added new functionalities for handling recursive dependency retrieval and loading steps from DAG files.

Bug Fixes

Adjusted the logic for selecting steps for archiving based on updated criteria, ensuring accurate step selection.

Tests

Introduced fixtures for mocking DAG structures and dependencies in tests.
Refactored tests to align with the updated logic and function names in the ETL process.

Hey @Marigold, walden unit tests are failing because of bad formatting of imports, but when I run make format locally nothing changes (and I don't know what the issue is). We could simply avoid checking the formatting of the ingest files. Do you have any suggestion? Thanks.

@pabloarosado Walden uses legacy formatting with isort / black and make format from etl directory doesn't affect it. You have to run cd lib/walden && make format to fix it.

Hi @Marigold thanks! I was actually running make format within lib/walden. But I was doing it from the etl poetry shell. Now I deactivated it and ran make format again from lib/waldenand... Now I'm having issues installing pyzmq and numpy together :-/ . Does poetry install work for you in walden? I'm trying a few things but I'm so far not succeeding.


Cache instance is resolved at import time rather than runtime

The cache to be used by django-cookie-consent is frozen at import time and makes it annoying to swap out cache backends during tests using django.test.override_settings.
Our default settings use the real cache backend, but in testing we're replacing this with the no-op cache backend. Except, because the cache instance is resolved at instance time, using override_settings has no effect. This means that during testing, test isolation is broken because there's now cache entries that span multiple tests.
https://github.com/bmihelac/django-cookie-consent/blob/510887228a5209d6355a99fca62bb6683d84b4d3/cookie_consent/cache.py#L6
I can resolve this for now using unittest.mock.patch, however, the get_cache call should really happen inside the functions exposed in the module rather than at import time.

@sergei-maertens thanks for report. I agree. Would you like to make a PR?

I would like to, but I'm afraid I have other priorities at the moment so if anyone else wants to pick it up, feelf ree!
As an unrelated item - I've recently sent an e-mail as well to the e-mail address in your Github profile for something that's not suited to be reported publicly, could you give that a look too?

@sergei-maertens I don't really know what you mean so it is hard for me and maybe some others to help. Can you provide some more code or even your console output to show us what you are talking about or can you explain it in an easier way? I am confused by runtime vs import vs instance time and what you mean by frozen. What is the no-op cache backend? It seems to be that you cannot switch to another cache backend like redis/memcache or others so easily? If you can make a PR that would be great because as of right now I am having trouble understanding.


feat(console): add wake-to-poll histogram

refactored MiniHistogram so wake to poll histogram can reuse some
code from poll duration histogram
added Percentiles widget so wake to poll can reuse code from poll
duration histogram

for https://github.com/tokio-rs/console/issues/50

This PR is quite long with the refactoring, I can split it into two if that's better for reviewing

This is excellent work, thanks for getting started here! I think one thing to work out that may affect this PR is whether we want a histogram per task, or if we just want the history of the whole runtime.
I'm tempted to say both sound interesting... The full runtime histogram can help show the overall "health" or busy level. If things are starting to take longer and longer to be polled, it could imply the system has too much load. But knowing at an individual task level could also help understand, when load got high, or another few tasks ate up precious CPU time, that it delayed waking some specific task.
By keeping a log of all the events and timestamps in the "recording", we could always reconstruct either of these. So probably we also want to consider the cost of building these up per tasks and sending them over the wire...
All of my rambling didn't have any decisions, just musings, but hopefully with some joint thinking we'll quickly arrive at a proper balance. :)

@seanmonstar thank you for the thoughts! I agree with you that recording this data for both runtime and per task is useful here.

So probably we also want to consider the cost of building these up per tasks and sending them over the wire...

This is probably my main concern in keeping logs of events. Given that HdrHistograms are pretty small, another option would be to have another histogram in each TaskStats. What do you think about that idea?

As this PR was not progressing, a new PR was opened (#409) which implemented the new histogram. That PR has been merged, so I'll close this one.


Issue14

Thanks for the quick response! Could you please paste a screenshot here to check how the error message looks like?

The error is thrown when there are no trailing digits after e, x, b, o. Otherwise the expression would unexpectedly evaluate to null.


Create AssetDBWriter, update TradingEnvironment and tests

DO NOT MERGE
@jfkirk @richafrank @ssanderson
TL;DR: I’m opening up this PR  as a means of soliciting feedback on the new write_data method of the TradingAlgorithm and the associated changes to the test suite.
Going forward the AssetFinder will exclusively be a reader class. Therefore we should deprecate the TradingEnvironment’s update_asset_finder method. In it’s place I have created a new method, write_data, which will write data to the underlying database using the AssetDBWriter class (using either the TradingEnvironment’s existing database engine, or the engine you specify in the write_data method).
This change in turn necessitates various changes across our test suite. Currently we write data to our underlying storage in a number of different places (which quickly becomes an issue with the enforcement of SQL UNIQUE constraints.)
Test suite changes
The creation of the AssetDBWriter creates an opportunity for some important changes to the test suite.
Remove unnecessary writing functionality from:

SpecificEquityTrades in zipline/sources/test_source.py
create_trade_history() in zipline/utils/factory.py
RandomWalkSource in zipline/sources/simulated.py

If a test wants to write data to the db, they must either 1) use the TradingEnvironment’s write_data method directly via:
trading.environment.write_data(equities_data = ..., futures_data = ...)

or 2) supply it in an instance of the TradingAlgorithm (however currently this only handles equity data)
TradingAlgorithm(asset_metadata=data)

If a test wishes to reuse sids (popular choices for testing are 0, 1 or 133) it it should instantiate a new trading environment by calling:
trading.environment = trading.TradingEnvironment()
This will initialize a new TradingEnvironment and with it a new empty in-memory database. This in particular is a point I would like feedback on. An alternative would be to add a method ‘update_engine’ to the TradingEnvironment, which would be less costly.
As an example of these changes taking effect I have updated tests/test_algorithm.py to be consistent with the updates - I would particularly appreciate feedback on the changes I've made to this file.
To do:

[x] Update outstanding tests
[ ] Refactor TradingAlgorithm to accept other data (e.g. futures), not just equities.

Self-assigning, as I am taking over this branch

@ssanderson @richafrank @ehebert This PR now contains a commit to remove the global TradingEnvironment and, by extension, the global AssetFinder. To compensate for this, the classes and methods that used the global env now accept the env/finder that it needs.
A TradingAlgorithm accepts an env as an arg. If an env is not provided, a default TradingEnvironment is build by the TradingAlgorithm, and all subclasses will use that TradingEnvironment.

Add class tear-downs the del the env
TradingEnvironment init kwargs for futures metadata
PositionTracker pickling unnecessary OrderedDicts
Cumulative risk metrics treasury curve serialization


Added custom default host and port.

Added custom default host and port for php artisan serve.

You cant use env() here. You need to use config() otherwise this will break for people using config:cache

I don't see why we'd need this set by env?

You could just alias the command?

Yes!
Thank you!
I close pull request.


⚠ Avatar API Endpoint has degraded performance

In 78c9e9b, Avatar API Endpoint (https://avatar.roblox.com/v1/avatar-rules) experienced degraded performance:

HTTP code: 200
Response time: 5011 ms

Resolved: Avatar API Endpoint performance has improved in b508b3f after 6 minutes.


Build status links in readme.md are broken

yeah I can't remember when I saw this working

fixed in https://github.com/Microsoft/visualfsharp/pull/2017


typos and flow issues in wiki page

I would love to be able to submit edits on the wiki when I find issues.  The Javascript styleguide, for example, has some misplaced code block endings leading to markdown showing up in the finished product.  https://github.com/FreeCodeCamp/FreeCodeCamp/wiki/Free-Code-Camp-JavaScript-Style-Guide#indent.  Can I help?
For example, space after function name section is actually part of the Curly Braces Everywhere section because of misplaced triple backticks.
space after function keyword expect in anonymous functions.

@ArbitraryHubris thanks for posting this issue!
If you want to contribute to our wiki you should fork https://github.com/FreeCodeCamp/wiki repository, make changes in your fork and then create a pull request. Before contributing, please read these articles:
https://github.com/FreeCodeCamp/FreeCodeCamp/wiki/Wiki-Style-Guide
https://github.com/FreeCodeCamp/FreeCodeCamp/wiki/wiki-js-template
https://github.com/FreeCodeCamp/FreeCodeCamp/wiki/Pull-Request-Contribute

Please review the Guidelines for Contributing, thank you!.

Thanks!  Closing.


MXnet - can install but not load

For bugs or installation issues, please provide the following information.
The more information you provide, the more likely people will be able to help you.
Environment info
Operating System: Windows 7 Professional
Compiler: R for Windows 3.3.2
Package used (Python/R/Scala/Julia): R for Windows 3.3.2
MXNet version: package ‘mxnet’ version 0.7
Or if installed from source:
MXNet commit hash (git rev-parse HEAD):
If you are using python package, please provide
Python version and distribution:
If you are using R package, please provide
R sessionInfo():
R version 3.3.2 (2016-10-31)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base
loaded via a namespace (and not attached):
[1] igraph_1.0.1       Rcpp_0.12.8        rstudioapi_0.6     magrittr_1.5       munsell_0.4.3
[6] colorspace_1.3-2   R6_2.2.0           brew_1.0-6         stringr_1.1.0      plyr_1.8.4
[11] dplyr_0.5.0        visNetwork_1.0.3   Rook_1.1-1         tools_3.3.2        grid_3.3.2
[16] gtable_0.2.0       DBI_0.5-1          influenceR_0.1.0   DiagrammeR_0.9.0   htmltools_0.3.5
[21] lazyeval_0.2.0     digest_0.6.11      assertthat_0.1     tibble_1.2         gridExtra_2.2.1
[26] RColorBrewer_1.1-2 ggplot2_2.2.1      htmlwidgets_0.8    viridis_0.3.4      rgexf_0.15.3
[31] stringi_1.1.2      scales_0.4.1       XML_3.98-1.5       jsonlite_1.2
Error Message:
Please paste the full error message, including stack trace.
I can install mxnet package but I can't make it available for use, problem with version 0.9.0 DiagrammeR

library("mxnet", lib.loc="~/R/win-library/3.3")
Error : object ‘combine_edges’ is not exported by 'namespace:DiagrammeR'
Error: package or namespace load failed for ‘mxnet’

Minimum reproducible example
if you are using your own code, please provide a short script that reproduces the error.
Steps to reproduce
or if you are running standard examples, please provide the commands you have run that lead to the error.

What have you tried to solve it?

Unistalled DiagrammeR_0.9.0 and tried to install earlier version with following commands

remove.packages("DiagrammeR")
install_version("DiagrammeR", version = "0.8.1", repos = "cran.us.r-project.org")
response to install_version command:
Warning: unable to access index for repository cran.us.r-project.org/src/contrib:
scheme not supported in URL 'cran.us.r-project.org/src/contrib/PACKAGES'
Error in package_find_repo(package, repos) :
couldn't find package 'DiagrammeR'

This has already been fixed in https://github.com/dmlc/mxnet/pull/4549
The Windows pkg hasn't be update yet.

Thanks for the update! Glad to hear about the progress.

From: Qiang Kou (KK<EMAIL_ADDRESS>Sent: Sunday, January 8, 2017 1:24 AM
To: dmlc/mxnet
Cc: pauldubose; Author
Subject: Re: [dmlc/mxnet] MXnet - can install but not load (#4574)
This has already been fixed in #4549https://github.com/dmlc/mxnet/pull/4549
The Windows pkg hasn't be update yet.

You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/dmlc/mxnet/issues/4574#issuecomment-271122424, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AX0FJpbTfRuicJHS7R1re7H3kKwJEPiEks5rQDrOgaJpZM4LdVvg.

I have the same problem. @thirdwing The MxnetR update for windows will be available soon? Thanks a lot for this fantastic package for R users!

Before we update the Windows pkg, you can try the following solution:
require(devtools)
install_version("DiagrammeR", version = "0.8.1", repos = "http://cran.us.r-project.org")

The latest version of DiagrammeR breaks many APIs.

Thanks for the code snippet
I ran the second line, but not the first line.  Good to know.
Paul

From: Qiang Kou (KK<EMAIL_ADDRESS>Sent: Monday, January 9, 2017 12:12 AM
To: dmlc/mxnet
Cc: pauldubose; Author
Subject: Re: [dmlc/mxnet] MXnet - can install but not load (#4574)
Before we update the Windows pkg, you can try the following solution:
require(devtools)
install_version("DiagrammeR", version = "0.8.1", repos = "http://cran.us.r-project.org")
The latest version of DiagrammeR breaks many APIs.

You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/dmlc/mxnet/issues/4574#issuecomment-271191914, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AX0FJiCnNy3rBDW3VrdPzkBATI8jJr_Tks5rQXt3gaJpZM4LdVvg.

Thx @thirdwing !

This has been fixed. Feel free to reopen if necessary.


convert problem

before convert

after convert

most of the cubes r rotated.

Attach your model file.
This extension was tested with more than 20 models made by various artists and the blocks rotations are untouched

crab_1.zip

This happens because you rotated the cubes and not the bones.
Cubes can't have cube rotated in such a way that the game items can't be rotated.
For example if you had created the crab directly as iaentitymodel you would have received this error on export:

What you have to do is use a single bone for each one of your cubes instead of using multiple cubes on a single bone.
I fixed the model for you, the only thing left to fix are the legs animations.
crab_1_fixed.zip


fix: import non-existant package

Fixes minor issue with importing a non-existent package where a part is the current package name.
Fixes #689

I dont know the test will ever pass go consistency, but the fix works.


Themes: Scoope Settings (Background)

Hi there
I'm working on a theme vor Visual Studio Code for our devs.
According to docs.sublimetext it should be possible to define the background Color of "scooped settings"...
Somehow VSC seems to ignore this constantly .
Could it be, VSC isn't allowing background colors for specific code-blocks?
I searched the web for bout 4 hours now and it seems that i can't find any reasons why this should not work at all.
My Code in myTheme.theme.json (made with Yo Code)
		{
			"name": "Comments",
			"scope": "comment",
			"settings": {
				"fontStyle": "bold",
				"foreground": "#155230",
				"background" : "#FFFFFF"

			}

I'd appreshiate  help or advice on this topic.
Regards
Roman
P.S.: If I should ask this question at another Forum, please let me know.

That's a known limitation of VSCode: Duplicate of #3429


Pydantic dataclasses cannot be used as Annotated Validators

Initial Checks

[X] I confirm that I'm using Pydantic V2

Description
Pydantic dataclasses cannot be used as Annotated Validators. The TypeError happens at class the definition.
I wanted to replicate the example of the section As a method on a custom type of the "Types" part of the Concepts but instead of using Python dataclasses, I wanted to use Pydantic dataclasses (to add validation).
I am not sure about all the internals but it looks like some conflicts about the __get_pydantic_core_schema__ that needs to be implemented for the class to be an Annotated Validator and the __get_pydantic_core_schema__ that is used for the definition.
I've read in the Pydantic website that [...] you should try to stick to the built-in constructs like those provided by annotated-types, pydantic.Field, or BeforeValidator and so on., so sorry if this is somehow already known and I should just avoid using Pydantic dataclasses in this specific context.
Example Code
import dataclasses
from typing import Annotated, Any

import pydantic
from pydantic import GetCoreSchemaHandler, PositiveFloat, TypeAdapter
from pydantic_core import CoreSchema, core_schema

@dataclasses.dataclass
class PythonDataclassValidator:
    exponent: PositiveFloat

    def __call__(self, value):
        return value**self.exponent

    def __get_pydantic_core_schema__(
        self, source_type: Any, handler: GetCoreSchemaHandler
    ) -> CoreSchema:
        """Return the Pydantic core schema for the custom after validator."""
        return core_schema.no_info_after_validator_function(self, handler(source_type))

### Error will happen at the next code line, during the class definition
### TypeError: PydanticDataclassValidator.__get_pydantic_core_schema__() missing 1 required positional argument: 'handler'

@pydantic.dataclasses.dataclass
class PydanticDataclassValidator:
    exponent: PositiveFloat

    def __call__(self, value):
        return value**self.exponent

    def __get_pydantic_core_schema__(
        self, source_type: Any, handler: GetCoreSchemaHandler
    ) -> CoreSchema:
        """Return the Pydantic core schema for the custom after validator."""
        return core_schema.no_info_after_validator_function(self, handler(source_type))

# This works as expected if I remove the whole code of the class PydanticDataclassValidator (this similar to the example given in Pydantic's website)
PythonType = Annotated[float, PythonDataclassValidator(2)]
print(TypeAdapter(PythonType).validate_python(2) == 4)

PydanticType = Annotated[float, PydanticDataclassValidator(2)]
print(TypeAdapter(PydanticType).validate_python(2) == 4)

Python, Pydantic & OS Version
pydantic version: 2.7.1
        pydantic-core version: 2.18.2
          pydantic-core build: profile=release pgo=true
                 install path: /home/leandro/miniforge3/envs/vbuild/lib/python3.10/site-packages/pydantic
               python version: 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0]
                     platform: Linux-6.8.0-76060800daily20240311-generic-x86_64-with-glibc2.35
             related packages: pydantic-settings-2.2.1 typing_extensions-4.11.0
                       commit: unknown

Hi @salemileandro,
Thanks for the question! Indeed, looks like a conflict on the __get_pydantic_core_schema__ front. I think this should be possible, but will definitely take some modification on the internal core schema building front.
I'll take a closer look at this for 2.8 and see if we can come up with a workaround for you!

Hi @salemileandro,
After chatting about this a bit more with the team, we've decided that we'd recommend just using plain dataclasses as annotated metadata for now.
We can revisit this for V3, but I'm going to remove this from the v2.8.0 milestone 👍

Hi!
Thanks for letting me know! It's far from being a blocking bug, I can use other ways to achieve similar goals so no worries :)

Hi! Thanks for letting me know! It's far from being a blocking bug, I can use other ways to achieve similar goals so no worries :)

Hi!
Could you show how to fix this bug pls? many thanks

Don't use the @pydantic.dataclasses.dataclass decorator. Use the standard python dataclass decorator. You won't have validation, but that's how it is.
If you really need/want validation in your class, you can always use a pydantic TypeAdapter within the post_init() method =)


Line break when line contains backslash

eg.
location ~ /\.ht {
deny all;
}
after formatted will look like this
location ~ /
\
.ht {
deny all;
#}

Hey thanks for reporting I will take a look at it shortly.

Hey , it seems to not happen at my nodejs port, maybe 1connect can use my repository to figure out why.
please try it here. I don't know python hope JS can be of help here.

@kum1ko seems to be fixed now.

Thanks @slomkowski


No "groups" icon

Material icons have groups icon but not in flutter

@Vurarddo The icon was added per this issue, https://github.com/flutter/flutter/issues/71233.
I verified this on latest master and see the same.

flutter doctor -v
[✓] Flutter (Channel master, 2.1.0-13.0.pre.376, on Mac OS X 10.15.4 19E2269
    darwin-x64, locale en-GB)
    • Flutter version 2.1.0-13.0.pre.376 at
      /Users/dhs/documents/Fluttersdk/flutter
    • Framework revision 1f19fd7108 (8 hours ago), 2021-03-31 21:32:49 -0700
    • Engine revision ec319c85be
    • Dart version 2.13.0 (build 2.13.0-187.0.dev)

[!] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.3, Build version 12C33
    ! CocoaPods 1.9.3 out of date (1.10.0 is recommended).
        CocoaPods is used to retrieve the iOS and macOS platform side's plugin
        code that responds to your plugin usage on the Dart side.
        Without CocoaPods, plugins will not work on iOS or macOS.
        For more info, see https://flutter.dev/platform-plugins
      To upgrade see
      https://guides.cocoapods.org/using/getting-started.html#installation for
      instructions.

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.54.3)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.20.0

[✓] Connected device (3 available)
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729 • ios
      • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                               •
      web-javascript • Google Chrome 89.0.4389.114

! Doctor found issues in 1 category.

You may need to switch to latest master to see it.  I am going ahead and closing this, since the icon is already added.


Prometheus metrics exporter (PUSH)

Enable prometheus metrics export as an alternative to OTEL. Given an endpoint, the auto-instrumenter will push the metrics there.

This seems out of scope for now, closing as note planned.


Verify datalayer

Migrated from: https://github.com/aemsites/esri/issues/149

I'm seeing page name, title, path local being pushed to dataLayer object.
Initial conversation with Analytics team -- we need to configure OneTrust before it will be pushed into the Abode datalayer.
closing as resolved for now.


For USERS only: regex match and min max lengths are not validated

Describe the bug
Adding a "regex match", and*or min and max lengths to users table are not validated before registering a new user.
Happens on saltcorn.com too
To Reproduce

-> whatever you enter as new user, it will pass.
Expected behavior
test for regex match

It works for me. There are other problem  - you cannot create user without role.

before registering ....

You're in the user settings, but if you tried validation at registration (using a custom page), it didn't work.

It works now


Quiet warnings and add ruby 2.4

Enable ruby warnings in this gem's specs, and check to see if the memoized ivar is defined before accessing it. This will quiet the flood of warnings in other projects that have warnings enabled.
Update the CI build matrix to remove unsupported rubies and add 2.4
Start returning a symbol from Memoizer.ivar_name, and some cleanup to support that.

🍰

💎 🦈


Second Issue.

Testing RPCs.

Testing comments API.

hi

Testing comments API.

Testing comments API.

Testing comments API.

Testing comments API.

hi

hi

hi

hi

hi

hi

hi

hi

hi

hi

test

test

test

test

test

test

Testing comments.

Testing comments.

Testing comments.

Testing comments.

Testing comments.

Testing comments.

Testing comments.

Testing comments.

Testing comments.

test

test


Why it's still old version on bower?

I want to get the updated and patched version (merged by pull request)

:+1: please update on bower

Updated to 0.0.5


FVD values of PVDM are strange

Hi, I am the first author of PVDM, and I just checked the FVD values of PVDM are much worse than the values that I reported in the paper. Could you tell me why such differences exist?
Many people tried (and succeeded) to reproduce the values, so it is weird to me.

Hi, I am the first author of PVDM, and I just checked the FVD values of PVDM are much worse than the values that I reported in the paper. Could you tell me why such differences exist?
Many people tried (and succeeded) to reproduce the values, so it is weird to me.

Hello, thanks for your interest of our work. For UCF101 and Skytimelapse datasets, we followed the paper and used provided pre-trained checkpoints for evaluation. However, we were unable to reproduce the reported results. It would be very helpful if you could provide any details or a complete evaluation code on how to obtain the results.

Just a quick check:

Did you use ema weights?
Did you use 2,048 samples for evaluation?
Did you use 400 steps (DDPM sampler) for generating samples?

Hi,
there even exists a recent arxiv paper that has succeeded to reproduce the values of PVDM: https://arxiv.org/abs/2402.13729v1
If you face difficulties to reproduce the values, you should ask authors before making it public..

Can you please reply to this issue?

Thank you very much for sharing the info and sorry for missing your question. We will thoroughly review the results with the information you've provided. We would greatly appreciate any additional details you could offer regarding the reimplementation of your work. Many thanks again.


[VR] Adjust text position

Hi, I have tried the HelloWorldVR.
At first, the position of the textNode is new Vector[0,0,-2]
However, after I changed the position into new Vector[0,0,2], I cannot see the "HELLO WORLD" text anymore. I expected the text will appear in the opposite position of the initial position.
Is it due to vr in viro core only supports 3DOF? Or is there a way to position the text in a VR scene in viro core?

Hey @HLneoh, this is because the text that is now placed behind you (at 0,0,2), you are effectively looking at the back side of the text geometry, and as such that geometry is being culled.
You can fix this by either billboarding the text to always face the user, or by setting disabling culling on the text's material.

Hi @dthian , Thanks for the comment. I have set to disable culling on text's materials as below:
           _Text helloWorldText = new Text.TextBuilder().viroContext(mViroView.getViroContext()).
            textString("Hello").
            fontFamilyName("Roboto").fontSize(40).
            color(Color.WHITE).
            width(4).height(2).
            horizontalAlignment(Text.HorizontalAlignment.CENTER).
            verticalAlignment(Text.VerticalAlignment.CENTER).
            lineBreakMode(Text.LineBreakMode.NONE).
            clipMode(Text.ClipMode.CLIP_TO_BOUNDS).
            maxLines(1).build();

           Material material = new Material();
           material.setCullMode(CullMode.NONE);
           helloWorldText.setMaterials(Arrays.asList(material));_

However, the text become all white and cannot show the word "Hello" in VR scene as the picture below.

I want to place text not only the area face to the user, but also the other areas in the 360 degree photo (e.g right hand side, left hand side and opposite side of area face to the user).
May I know what way can I use to achieve it?

Hi @HLneoh, have you tried the billboarding the text as mentioned above?

Hi @dthian , it works after billboarding the text. Thank you so much. :)


Show indication when training data has changed since the model was last trained.

i.e. indicate a "edited" state requiring re-training, similar to showing files that have been edited and not saved.

Probably show this indication in the prediction (by graying the text?).
From the workflow, after adding new samples, people might expect the prediction results improved. At that stage, their attention is mainly on the prediction.

I was thinking of something like inserting an asterisk (*) before the name of the class label, which I feel like is something I've seen in other editors. Or maybe the word "edited" after the label (which might mean on the next line)?

See https://github.com/damellis/sensors/commit/1db6f025edab24a9fa29a4ca7704b4465f6270ad.
Note that adding samples followed by deleting it is considered "edited".


postgres: add log to specify Agent uses password or token to connect to db

What does this PR do?

Add log to report Agent uses password or token for connection.
Motivation

For easy troubleshooting.
Agent replaces a given password with token implicitly when aws / azure section is configured.

Additional Notes

2024-08-13 05:14:17 UTC | CORE | INFO | (pkg/collector/python/datadog_agent.go:131 in LogMessage) | postgres:8343993ddd1c5f02 | (postgres.py:796) | Try to connect to <IP_ADDRESS> with password

Review checklist (to be filled by reviewers)

[ ] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
[ ] Changelog entries must be created for modifications to shipped code
[ ] Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
[ ] If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

This is why I chose log.info.

This log should not be a spam. This line doesn't hit after Agent can successfully connect to the db.
Reduce the number of messages between DD and customers. Support Engineers don't need to ask customers to set debug mode.


[machinst x64]: implement dot product

Implements dot product for the x64 backend. Probably should not be merged until we can enable the spec tests, which requires https://github.com/WebAssembly/testsuite/pull/37 to merge (merged in #2481) and https://github.com/bytecodealliance/wasmtime/issues/2432 to be resolved.

Duplicated in #2530.


How to use vscode debugger?

Hi,
this extension seems like a great way to start using modal editing :)
After working through some tutorials on how to use vim, I feel more or less comfortable with editing text.
Now I'd like to use this for actually developing something.
Is there a way to use the vscode debugging functionality?
How do I set a breakpoint, step, continue, run to cursor, jump to eval terminal, ... ?
Thanks for developing this amazing plugin :)

I'm not the developer. But debugging should work exactly like it does without the neovim extension. Every resource on debugging with VSCode should apply. Is this not the case with you?

A bit late, but here's a snippet that I use to control the debugger:
nnoremap <silent> <leader>R <Cmd>:call VSCodeNotify('workbench.action.debug.start')<CR>
nnoremap <silent> <leader>b <Cmd>:call VSCodeNotify('editor.debug.action.toggleBreakpoint')<CR>
nnoremap <silent> c <Cmd>:call VSCodeNotify('workbench.action.debug.continue')<CR>
nnoremap <silent> s <Cmd>:call VSCodeNotify('workbench.action.debug.stepOver')<CR>

<leader>-Shift-r start new debug session
<leader>-b toggle breakpoint
c continue execution
s step over the current instruction

Where <leader> is , in my Vim but I believe it is \ by default. All of these commands only work in normal mode.
You get the idea. Feel free to edit it to suit your needs. There might be a way to use c and s for something else but it would require something more complex that checks whether there is a debug session running. Not sure if that is feasible with the current API.

Closing this issue. If something is not clear let me know.


refactor: map feature toggle

This PR addresses #4507

[x] Addresses the issue in full
[ ] Addresses only certain aspects of the issue

Description
Adds back in the old map files, postfixed with Deprecated.
Adds a new dynamic feature toggle to turn the map experience on / off.
On desktop:
Keeps the UI changes from the new map to have a full screen and new search bar, but uses the old map / only fetches 25 listings
On mobile:
Reverts back to the swipe-up experience, but keeps the full screen map and the new search bar.
Additionally, when the toggle is on, fixes a bug where externally fetched listings weren't showing in info modals after clicking on a pin.
How Can This Be Tested/Reviewed?
With SHOW_ALL_MAP_PINS as false, compare the listings map experience to prod. We will have the full screen maps and placement of the search bar, but the rest should be very similar.
There's lots of lines added from copying and pasting the old files. These Deprecated files have no changes to just a line or two to support toggling. The files were already messy, so I wouldn't focus time there since we're moving away from that version.
Author Checklist:

[x] Added QA notes to the issue with applicable URLs
[x] Reviewed in a desktop view
[x] Reviewed in a mobile view
[x] Reviewed considering accessibility
[ ] Added tests covering the changes
[ ] Made corresponding changes to the documentation
[ ] Ran yarn generate:client and/or created a migration when required

Review Process:

Read and understand the issue
Ensure the author has added QA notes
Review the code itself from a style point of view
Pull the changes down locally and test that the acceptance criteria is met
Either (1) explicitly ask a clarifying question, (2) request changes, or (3) approve the PR, even if there are very small remaining changes, if you don't need to re-review after the updates

The recent changes were to fix a bug where the swipeable area on mobile wasn't always sticky to the top


Disable strict header check

HTTP/2 200
cache-control: no-cache, no-store, must-revalidate, pre-check=0, post-check=0
content-disposition: attachment; filename=json.json
content-length: 2263
content-type: application/json;charset=utf-8
date: Tue, 12 Mar 2019 02:20:17 GMT
expires: Tue, 31 Mar 1981 05:00:00 GMT
last-modified: Tue, 12 Mar 2019 02:20:17 GMT
pragma: no-cache
server: tsa_m
status: 200 OK
strict-transport-security: max-age=631138519
x-access-level: read-write-directmessages
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-rate-limit-limit: 900
x-rate-limit-remaining: 889
x-rate-limit-reset:<PHONE_NUMBER>
x-response-time: 133
x-twitter-response-tags: BouncerCompliant
x-xss-protection: 1; mode=block; report=https://twitter.com/i/xss_report

https://tools.ietf.org/html/rfc7540#section-<IP_ADDRESS>
   For HTTP/2 responses, a single ":status" pseudo-header field is
   defined that carries the HTTP status code field (see [RFC7231],
   Section 6).  This pseudo-header field MUST be included in all
   responses; otherwise, the response is malformed (Section <IP_ADDRESS>).

   HTTP/2 does not define a way to carry the version or reason phrase
   that is included in an HTTP/1.1 status line.

patched https://github.com/39ff/cowitter/commit/96758adef97e121bfe7755be0dbb5a2346955be3


B-18648, B-18855, and B-18851

There was some overlap between changes on the PRs associated with B-18855 and B-18648.
I have included changes on PRs associated with B-18851 because that story falls under the same feature as B-18648.
PR #12449 link
PR #12428 link
PR #12372 link
PR #12325 link
PR #12461 link
PR #12443 link
PR #12352 link
GREP string might be something like this:
--grep="\(12449\|12428\|12372\|12325\|12461\|12443\|12352\)"

verify rpt:
B18648_B18855_B18851-12517.verify_rpt.txt
there were diffs

verify rpt: B18648_B18855_B18851-12517.verify_rpt.txt
there were diffs

~~I just made it through all of these diffs. I feel like either the diffs are inaccurate at one point or I missed one of the PRs that includes some of the changes.
That being said, I 100% certify that these changes are all in the ingegrationTesting branch and were all approved through the POA process.~~
After discussing with Tae, he showed me at least one spot where there's a concerning diff (integrationTesting includes a change that has to do with these Agility tickets that isn't included in this commit). I will be digging a little deeper on this and will try to rectify this.

Cool.  :)


XMLRPC PECL Extension Not Loading

The PECL XMLRPC extension is being built but it is not being enabled in PHP.
May need to explore using an alternative: https://php.watch/versions/8.0/xmlrpc

There are no longer any real compatible PECL modules. If this functionality it ever needed a Composer package will need to be installed and leveraged.


PKIX path building failed: (IBMCertPathBuilderException) unable to find valid certification path to requested target

❗ please do not add sensitive information in issues, you can provide extra information via email using issue number as reference ❗
Describe the issue
At random, the HttpSender runs into the following error, whilst the certificate is present in the store:
PKIX path building failed: (IBMCertPathBuilderException) unable to find valid certification path to requested target
The certificate is 100% present, here is a screenshot from WAS:

Once again verified with a custom pipe to list all certs:

I decided to create a custom component which should send HTTPS in the most basic form possible. Since last evening I am running a test where every minute I test three implementations:

Basic HttpSender (name of job = Test connection login.microsoftonline.com proxy)
Extended HttpSenderBase as plugin for MSAL (name of job = receivemails) see #3368
Custom HTTPS component (name of job = Test connection SSL)

Their results:

Basic HttpSender

Extended HttpSenderBase

Custom HTTPS

Ladybug logging show all jobs, as you can see the HTTP sender based job does 'restore' it self after a bunch of errors.

Confirmed by test a pipeline that the basic HTTPS version kept working whilst the HttpSender broke:

Sender configuration:
<pipe className="nl.nn.adapterframework.pipes.SenderPipe" name="Test">
	<sender authAlias="some-alias" className="nl.nn.adapterframework.http.HttpSender" methodType="GET" name="test" proxyAuthAlias="some-alias" proxyHost="some-host" proxyPort="some-port">
		<param name="url" sessionKey="originalMessage"/>
	</sender>
</pipe>

Custom list certificates pipe (quick n dirty)
package nl.nn.adapterframework.pipes;

import lombok.Getter;
import lombok.Setter;
import nl.nn.adapterframework.core.PipeLineSession;
import nl.nn.adapterframework.core.PipeRunException;
import nl.nn.adapterframework.core.PipeRunResult;
import nl.nn.adapterframework.stream.Message;
import nl.nn.adapterframework.xml.SaxElementBuilder;

import java.io.*;
import java.security.KeyStore;
import java.security.cert.Certificate;
import java.util.Enumeration;

public class CertificatePipe extends FixedForwardPipe {
    private @Getter @Setter String password;
    private @Getter @Setter String location;

    @Override
    public PipeRunResult doPipe(Message message, PipeLineSession pipeLineSession) throws PipeRunException {
        InputStream keystoreLocation = null;
        try {
            keystoreLocation = getLocationAsInputStream();

            KeyStore keystore = KeyStore.getInstance(KeyStore.getDefaultType());
            keystore.load(keystoreLocation, getPassword().toCharArray());

            Enumeration<String> enumeration = keystore.aliases();
            SaxElementBuilder certificates = new SaxElementBuilder("certificates");
            while(enumeration.hasMoreElements()) {
                String alias = enumeration.nextElement();
                if(log.isDebugEnabled()) log.debug("alias name: " + alias);

                Certificate certificate = keystore.getCertificate(alias);
                if(log.isDebugEnabled()) log.debug(certificate.toString());

                certificates.addElement("certificate", "name", alias, certificate.toString());
            }

            Message result = new Message(certificates.toString());
            return new PipeRunResult(getSuccessForward(), result);
        } catch (Exception e) {
           throw new PipeRunException(this, "Could not list certificates because of exception", e);
        } finally {
            if(keystoreLocation != null){
                try {
                    keystoreLocation.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
        }
    }

    private InputStream getLocationAsInputStream() throws FileNotFoundException {
        File file = new File(getLocation());
        return new FileInputStream(file);
    }
}

Custom HTTPS impl (quick n dirty)
package nl.nn.adapterframework.pipes;

import lombok.Getter;
import lombok.Setter;
import nl.nn.adapterframework.core.PipeLineSession;
import nl.nn.adapterframework.core.PipeRunException;
import nl.nn.adapterframework.core.PipeRunResult;
import nl.nn.adapterframework.stream.Message;
import nl.nn.adapterframework.util.CredentialFactory;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.auth.AuthScope;
import org.apache.http.auth.Credentials;
import org.apache.http.auth.UsernamePasswordCredentials;
import org.apache.http.client.CredentialsProvider;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpRequestBase;
import org.apache.http.impl.client.BasicCredentialsProvider;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.ssl.SSLContexts;

import javax.net.ssl.SSLContext;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.InputStream;
import java.security.KeyStore;

public class MSALPipe extends FixedForwardPipe {
    private @Getter @Setter String password;
    private @Getter @Setter String location;
    private @Getter @Setter String proxyHost;
    private @Getter @Setter String proxyAuthAlias;
    private @Getter @Setter String proxyUsername;
    private @Getter @Setter String proxyPassword;
    private @Getter @Setter String authAlias;

    private @Getter @Setter int proxyPort;

    @Override
    public PipeRunResult doPipe(Message message, PipeLineSession pipeLineSession) throws PipeRunException {
        Message result = null;
        try {
            KeyStore keyStore = getKeyStore();

            SSLContext sslContext = SSLContexts.custom()
                    .loadKeyMaterial(keyStore,  getPassword().toCharArray())
                    .build();

            CredentialsProvider credentialsProvider = new BasicCredentialsProvider();
            CredentialFactory auth = new CredentialFactory(getAuthAlias(), null, getPassword());
            credentialsProvider.setCredentials(new AuthScope(AuthScope.ANY_HOST, AuthScope.ANY_PORT),  new UsernamePasswordCredentials(auth.getUsername(), auth.getPassword()) );

            CredentialFactory pcf = new CredentialFactory(getProxyAuthAlias(), getProxyUsername(), getProxyPassword());
            HttpHost proxy = new HttpHost(getProxyHost(), getProxyPort());
            AuthScope scope = new AuthScope(proxy, null, AuthScope.ANY_SCHEME);

            if (StringUtils.isNotEmpty(pcf.getUsername())) {
                Credentials credentials = new UsernamePasswordCredentials(pcf.getUsername(), pcf.getPassword());
                credentialsProvider.setCredentials(scope, credentials);
            }

            HttpClient httpClient = HttpClients.custom().setSSLContext(sslContext).setProxy(proxy).setDefaultCredentialsProvider(credentialsProvider).build();
            HttpRequestBase requestBase = new HttpGet(message.asString());
            HttpResponse response = httpClient.execute(requestBase);

            result = new Message(response.getEntity().getContent());

            return new PipeRunResult(getSuccessForward(), result);
        } catch (Exception e){
            throw new PipeRunException(this, "Could not connect to microsoft!", e);
        }
    }

    private KeyStore getKeyStore() throws PipeRunException {
        InputStream keystoreLocation = null;
        try {
            keystoreLocation = getLocationAsInputStream();

            KeyStore keystore = KeyStore.getInstance(KeyStore.getDefaultType());
            keystore.load(keystoreLocation, getPassword().toCharArray());

            return keystore;
        } catch (Exception e){
            throw new PipeRunException(this, "Could not get keystore", e);
        }
    }

    private InputStream getLocationAsInputStream() throws FileNotFoundException {
        File file = new File(getLocation());
        return new FileInputStream(file);
    }

}

Reporter
Laurens
Environment
FF! 7.7: Ibis4Input
Running on XX.XXXX. using IBM WebSphere Application Server/<IP_ADDRESS>
Java Version: Java(TM) SE Runtime Environment (<IP_ADDRESS> - pxa6480sr7-20211025_01(SR7))
Heap size: 1157M, total JVM memory: 4096M
Free disk space: 9GB, total disk space: 12GB
Up since: 2022-06-16 23:20:08 (15h)

After the weekend:
HttpSender:

Simple impl:

Extended httpSender (MSAL):

Another pic sharing results, in 2 days it happened only 20 times...

Only 1% of the time this happened the past few days, where as the simple impl has a 0% error rate.
Its very interesting to observe that the connection was working, after reloading the config, it broke:

I'm closing this issue as it's been stale for a while now.


Use same lnd's cert package to create TLS config to fix TLS cipher suites

We need to use safe TLS1.2 cipher suites that are also accepted by the golang http2 library. Turns out, that is a subset of what we currently use in lnd.

We ended up disabling TLS to get it to work. There seems to be a general problem with TLS and load balancers.
I decided to leave all the TLS changes in this PR anyway as this still might come in handy later on when we want to run aperture as a Kubernetes Ingress Controller.

We tested this again on the AWS cluster and confirmed it still works.


Fix list_cursor traceback and remove triggers.

Merged, thanks @showell!  It's at least a nice code cleanup to get rid of that event trigger.


Errors when launching the streamlit apps

First thank you for this cool project. I am having a problem getting to work. Note that all of the underlying streamlit apps work find when run from the CLI.
I issue the following command to launch the streamlit-launchpad and that works:

It shows the web page launcher:

But when I click on any app to run I get this error: (browser)

The command line:

I am using tornoda v5.1.1
Any help is appreciated. thanks!

Thank you for your interest!
Unfortunately, a component of Tornado does not work under Windows. In particular, this code uses tornado.process.Subprocess. The documentation there says it doesn't work on Windows.
If easy for you to run in a Docker container, that should work.
Or it should be possible to modify the code to run on Windows (probably just remove the line 79 that I linked above), but it wouldn't tidy up the processes when they finish.
Maybe we can see if these ideas help you, and then I'll add a note for Windows users on the readme.

Oh, I completely missed the part about not working on Windows. Thanks again for a great package

FYI, I submitted a new PR #8 based on your suggestions. tested on my Windows machine and it works. Thanks!

Thank you very much - I've released a version 0.0.11!
I still think Streamlit needs to build in this 'launchpad' functionality natively. I can't believe they haven't yet really!

I completely agree!
Can you push the v0.11 to pypi, thanks!

Oh yes, I missed that step... it's on there now.