Create automated functional tests (using Reqnroll) to verify Weevil's software requirements.

HowTo: Write Gherkin Scenarios
Scenario Titles
There isn't a single "one-size-fits-all" approach to naming Gherkin scenarios, but there are best practices that can make scenarios more understandable and maintainable. These practices aim to make the scenario name descriptive enough to understand its purpose at a glance while being concise enough to be easily digestible. Here are some guidelines:

Clarity Over Brevity: The name should make it immediately clear what the scenario is testing. While brevity is desired, it should not come at the expense of clarity.
Domain Language: Use the language that the team and stakeholders understand. Domain-Specific Language (DSL) can be very effective here.
Outcome-Oriented: Include the expected outcome or behavior in the scenario name to give a clear idea of what success looks like for the test.
Avoid Implementation Details: Try not to include how the test will achieve its goal but rather focus on what you are trying to achieve.
Consistency: Stick to a naming convention that the team agrees upon to make it easier to search for and understand scenarios.

Examples following these guidelines might look like:

"Successful login with valid credentials"
"Error on empty shopping cart checkout"
"Admin access denied for regular users"
"Open log file with valid format"
"Error on opening corrupted log file"

Lessons Learned

Single Quote / Double Quote / Backtick text formatting works for...

GitHub : single & double quoted text
GitHub codespace: does not support syntax highlighting for Gherkin strings

Relates To

#362

Articles

Powering your Cucumber Acceptance Tests with Jira Integration

Related to:

#325

Troubleshooting: DatenLokator Won't Run
Symptoms:

MS Test automated tests which include  SpecFlow + DatenLokator won't start via Visual Studio's Test Explorer.

Implementation Notes:

If you Debug a test you will discover that MS Test does in fact start.  The problem is:

You cannot use _MS Test+'s AssemblyInitialize attribute in conjunction with SpecFlow's test run hooks.

Solution:

Remove _MS Test+'s AssemblyInitialize.
Use SpecFlow's test run hooks to initialize the DatenLokator v2.0.0 library.

Challenges
STA Thread
The OpenAsync Gherkin scenario is failing with a run-time error:

InvalidOperationException: The calling thread must be STA, because many UI components require this.

With Just My Code disabled, you can see that the error is ultimately due to a .NET guard failing:
Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Utilities.VSInstallationUtilities.cs:
public static string? VSInstallPath
{
   // ...
   // Use the Setup API to find the installation folder for currently running VS instance.
   if (new SetupConfiguration() is ISetupConfiguration setupConfiguration)
   {
      var currentConfiguration = setupConfiguration.GetInstanceForCurrentProcess();
      // ...
   }
}

Next Steps

Is Weevil really running UI for the MS Test... or is this simply a WPF guard that is being accidentally triggered?
Solutions:

Avoid calling the guard, or
Run the functional test using a STA Thread

Related Articles

MSTest v2: Customize test execution

Talks about STA threads

Problem: Unexpected SpecFlow Errors
Symptoms
Visual Studio is displaying weird compiler errors & warnings

AndDiscoveryProviderSucceed: Error during binding discovery
FileNotFoundException: Could not load file or assembly 'System.Runtime, Version=<IP_ADDRESS>
JsonReaderException: Unexpected character encountered while parsing value
AndDiscoveryProviderSucceed: The project bindings (e.g. step definitions) could not be discovered. Navigation, step completion and other features are disabled.

In Visual Studio:

Gherkin statements that clearly did not exist... were not being displayed as purple text in Visual Studio.

Root Cause

This problem appeared after upgrading projects to .Net Core 8

Solution

Install the Visual Studio SpecFlow extension for .Net Core 8

Implementation
After porting from SpecFlow to Reqnroll, the run-time starting throwing the following exception because the dependency injection is slightly different with Reqnroll.
Solution:
* `BeforeTestRun` only accepts a `ITestRunnerManager` parameter
* `AfterTestRun` only accepts a `ITestRunnerManager` parameter

Problem:

Reqnroll.BoDi.ObjectContainerException
  HResult=0x80131500
  Message=Interface cannot be resolved: Reqnroll.ITestRunner
  Source=Reqnroll
  StackTrace:
   at Reqnroll.BoDi.ObjectContainer.TypeRegistration.<>c__DisplayClass3_0.<ResolvePerContext>b__1()
   at Reqnroll.BoDi.ObjectContainer.RegistrationWithStrategy.ExecuteWithLock(Object lockObject, Func`1 getter, Func`1 factory, ResolutionList resolutionPath, TimeSpan timeout)
   at Reqnroll.BoDi.ObjectContainer.TypeRegistration.ResolvePerContext(ObjectContainer container, RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at Reqnroll.BoDi.ObjectContainer.RegistrationWithStrategy.Resolve(ObjectContainer container, RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at Reqnroll.BoDi.ObjectContainer.ResolveObject(RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at Reqnroll.BoDi.ObjectContainer.Resolve(Type typeToResolve, ResolutionList resolutionPath, String name)
   at Reqnroll.BoDi.ObjectContainer.Resolve(Type typeToResolve, String name)
   at Reqnroll.Infrastructure.TestObjectResolver.ResolveBindingInstance(Type bindingType, IObjectContainer container)
   at Reqnroll.Infrastructure.TestExecutionEngine.ResolveArgument(IObjectContainer container, IBindingParameter parameter)
   at Reqnroll.Infrastructure.TestExecutionEngine.<>c__DisplayClass54_0.<ResolveArguments>b__0(IBindingParameter p)
   at System.Linq.Enumerable.SelectArrayIterator`2.Fill(ReadOnlySpan`1 source, Span`1 destination, Func`2 func)
   at System.Linq.Enumerable.SelectArrayIterator`2.ToArray()
   at Reqnroll.Infrastructure.TestExecutionEngine.ResolveArguments(IHookBinding hookBinding, IObjectContainer currentContainer)
   at Reqnroll.Infrastructure.TestExecutionEngine.<InvokeHookAsync>d__50.MoveNext()
   at Reqnroll.Infrastructure.TestExecutionEngine.<FireEventsAsync>d__48.MoveNext()
   at Reqnroll.Infrastructure.TestExecutionEngine.<FireEventsAsync>d__48.MoveNext()
   at Reqnroll.Infrastructure.TestExecutionEngine.<OnTestRunStartAsync>d__32.MoveNext()
   at Reqnroll.TestRunner.<OnTestRunStartAsync>d__10.MoveNext()
   at Reqnroll.TestRunnerManager.<FireTestRunStartAsync>d__35.MoveNext()
   at Reqnroll.TestRunnerManager.<OnTestRunStartAsync>d__46.MoveNext()
   at BlueDotBrigade_Weevil_MSTestAssemblyHooks.<AssemblyInitializeAsync>d__0.MoveNext() in C:\Code\BlueDotBrigade\OpenSource\weevil\Tst\BlueDotBrigade.Weevil.Core-FunctionalTests\obj\x64\Debug\net8.0\MSTest.AssemblyHooks.cs:line 25

  This exception was originally thrown at this call stack:
    [External Code]
    BlueDotBrigade_Weevil_MSTestAssemblyHooks.AssemblyInitializeAsync(Microsoft.VisualStudio.TestTools.UnitTesting.TestContext) in MSTest.AssemblyHooks.cs


disabledFields

Disable fields
->disableFields(['email', 'name'])

would you like to explain the use case please ?

would you like to explain the use case please ?

I have a use case, showing the user their role or which team they belong to, without being able to change it.
It's something I wanted to do and I couldn't do it.

Actually, you can solve this by creating CustomComponentes
https://github.com/joaopaulolndev/filament-edit-profile?tab=readme-ov-file#custom-components
and to hide the default fields you can use this method
->shouldShowEditProfileForm(false)
@KamranBiglari @Abdelhalim-Merabet @javier-villatoro


Updates for LH 084

Hi @RyanGlScott -
I made a bunch of updates to make the Sum/Prod files compatible with LH 084, so
so the UCSC folks @lkuper and @KamalaRamas can play around with those proofs.
Not sure if we want to merge this back into the master or create a separate branch
on your original? Any thoughts?
The relevant branch is here:
https://github.com/ranjitjhala/verified-instances/tree/lh-84

I have no objections to merging this into master—if it doesn't work with the latest version of Liquid Haskell, then let's fix it!

Well its not so easy because the rest of the stuff is linked to earlier versions of lvars etc. so not sure if that will break or no 😟

Ah. Is the issue that the LVar-related code doesn't build with a newer GHC? If so, perhaps we should take the existing stack.yaml file, rename it to something else (e.g., stack-lts-8.21.yaml), and add a new stack.yaml file without support for the LVar stuff. That way, we can incrementally port over the LVar code later, while still retaining the ability to build it if you explicitly point stack to the right YAML file.

Yes, I'd say either:

Just leave the current thing in a branch OR
First update the whole thing so it builds with stack-lts-12.2 or similar and then update the LH bits?

I can totally understand if you don't want to undertake the hassle of (2) ATM (other priorities etc.) in which case make an LH-84 branch or such and I'll submit a PR against that?

Question: is the new code backwards-compatible with old versions of Liquid Haskell? If so, I would suggest just keeping all of the code in master, since it wouldn't hurt anything to do so. If breaking changes are required, then I would be in favor of archiving the current code in a branch, then putting the new code in master.

Nope, the new code is not backwards compatible so it will break stuff.

so yes, better to archive I think and make the new stuff master? shall I submit a PR against master then?

Indeed, a PR against master sounds good. In fact, #11 has appeared. Hooray!
Also, I've archived the current master in the https://github.com/iu-parfunc/verified-instances/tree/lts-8.21 branch.

Resolved in #11.


How to use client-side session

Currently in the repo i only see an implementation for using auth() which can only be used on server side components.
How would I go about using the session hook "useSession" while keeping type inference. When i try to use just the import from next-auth I lose my custom user object. Same thing happens when i export useSession from the index.ts.
I would love to see how one would go about implementing this in a monorepo

It looks like the SessionContextValue returned from useSession is using the type from @auth/core/types, and not from next-auth. This means our module augmentation of next-auth wont have any effect on the useSession hook.
To fix this, you need to augment @auth/core/types too :/
  declare module "next-auth" {
    interface Session {
      user: {
        id: string;
      } & DefaultSession["user"];
    }
  }

+ declare module "@auth/core/types" {
+   interface Session {
+     user: {
+       id: string;
+     } & DefaultSession["user"];
+   }
+ }

I wonder if we could get next-auth to change so the useSession hook uses the type from next-auth instead.
Screenshot

Although testing it out again it does work when just augmenting next-auth for me?

Hmm.. you make a good point about moving session handling to the RSC. But i'll check to to maybe modify the type for now while i work on a migration plan to move my ui components to use server session.
Thank you

@shayant98 I did exactly that myself, making all session calls come from the RSC.
https://github.com/dBianchii/kodix-turbo/pull/40
This does mean you need to drill the session deep into your client components always, but is better than needing to fetch from client side after initial load.

If you have a deep tree of CCs you could mount a SessionProvider and pass the initialSession prop from the RSC session and then use useSession below. This will avoid drilling the prop too deep whilst not having the useSession data flicker, but for just one or two layers i dont think it’s an issue to pass as prop

If you have a deep tree of CCs you could mount a SessionProvider and pass the initialSession prop from the RSC session and then use useSession below. This will avoid drilling the prop too deep whilst not having the useSession data flicker, but for just one or two layers i dont think it’s an issue to pass as prop

Do you mean a local session provider for a part of the tree or a global in layout.tsx?

No a ”local” one if you have a part of the tree that’s deep in CCs - I wouldnt await the session at the top layout
maybe there’s a better way to have a global one without awaiting but as i said i havent had the issue where I’ve drillen the session deep enough where i felt it was necessary


Added beezlabs as an adaptor

Hey, I just made a Pull Request!
Added beezlabs as an adaptor of backstage
:heavy_check_mark: Checklist
Change to the ADOPTERS.md

Hi, good to see you! You'll see the DCO build details noting that you have to sign each commit made to the repo - as in, the signature has to go in the actual git history, not just in the github comments or similar. Let us know if we can help guide you through it more!

Hi, @freben thanks for the info,  I will close this request and raise a new PR with the sign-off


Remove code duplication in CommitLog putMessage ayncPutMessage methods

The methods asycPutMessage and putMessage in class CommitLog shares a lot of code:
https://github.com/apache/rocketmq/blob/b4240d5cea8d001c21b9c0d73f5aa700fcd0d568/store/src/main/java/org/apache/rocketmq/store/CommitLog.java#L559-L662
https://github.com/apache/rocketmq/blob/b4240d5cea8d001c21b9c0d73f5aa700fcd0d568/store/src/main/java/org/apache/rocketmq/store/CommitLog.java#L788-L902
removing duplication simplify the code base.

@andsel would you like to submit a PR to polish it?

@duhenglucky the PR #2854 solves this, missed to link it here


Remove 1.13 stuff in documentation stable branch

Describe the bug
On our documentation repo, develop got merged/rebased/cherrypicked into the hotfix/1.12.2 branch. There's 99% chance I'm the guy who did this, so I'm going to be the guy who fixes it. The rebase/merge/whatever doesn't break anything, but it does include a few small details about the next release (such as which version of Cromwell 1.13 will be using).
To Reproduce
You can see 1.13 referenced in https://docs.dockstore.org/en/stable/advanced-topics/dockstore-cli/advanced-features.html?highlight=cromwell#wdl-launcher-configuration
Expected behavior
~Ash is banned from using git rebase~ 1.13 should not be referenced on master/stable
Additional context
I found this issue this afternoon. Charles and I did some digging, using the addition of images to index.rst (which should be on only develop due to not being on 1.12's release branch) as our clue. Charles found the comment "merged commit 9748946" in https://github.com/dockstore/dockstore-documentation/pull/206, so that's probably where the rebase happened. Knowing me, I did it to bring in a linkcheck fix.
┆Issue is synchronized with this Jira Story
┆friendlyId: DOCK-2206
┆sprint: Backlog
┆taskType: Story

here's 99% chance I'm the guy who did this, so I'm going to be the guy who fixes it.

:grin:

@denis-yuen I tried, but I don't have the permissions required to do this in a timely manner. The change to be made is just this. I would recommend someone with the proper permissions just make this commit on stable and push it. It could be done with our hubflow tooling but it will require you immediately undo the commit on develop again.


Fix one_of, none_of, and char when meeting UTF-8 chars

As discussed on IRC, unsure if you're ok with adding a method to the AsChar trait, but at least it works.

Coverage decreased (-0.004%) to 87.953% when pulling 0640736bd50f12c02b7eba6c1c73f1e4b07ae1ce on meh:fix-utf8 into 20ee95d628b2a5483d743337e710b4245ec53a16 on Geal:master.


update handler screenshot (Java 11 -> 17)

The screenshot describing the Lambda handler config was outdated.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@deki you need to sign the CLA.

CLA signed. Then it looks like additional changes are required as MicronautLambdaHandler is referenced everywhere in the docs.

Updated it to ApiGatewayProxyRequestEventFunction.
Where do you plan to mention ApplicationLoadBalancerFunction and APIGatewayV2HTTPEventFunction? On the same page or as separate guides?

Where do you plan to mention ApplicationLoadBalancerFunction and APIGatewayV2HTTPEventFunction? On the same page or as separate guides?

We should create a common snippet and mention the different handler options in every guide.


feat: Feat/refactor to stages

Fixes #

So the issue I have found is that the opinionated "build" step of projen's fails when aws-cdk-github-actions is involved.
I get the error here:
https://github.com/michael-lowe-nz/league-lobster-text-reminders/actions/runs/4936407150/jobs/8823887493?pr=53#step:4:19
Error: "account" and "region" are required
Because these are unset in the build workflow, it fails.
So I've ended up hard coding in the account and region, and now this can't really be deployed to another account for now


added image cache

change alpa of notification view
added image cache

Why did you change alpha?

ignore alpha changes

if you cut "@objc" is not swift4 compatible

I have fixed swift 4 compatible issue.


SSR with wrapped import

💬 Questions and Help
Hello!
Im trying to add SSR support for existing project with async components. Our components should be loaded with some delays like described in https://loadable-components.com/docs/delay/: delay before starting network request (delayed import execution) and minimal time for fallback show.
While it works fine in CSR, SSR breaks with error loadable: SSR requires @loadable/babel-plugin, please install it despite @loadable/babel-plugin usage.
To avoid this problem we introduce env param which doesn't wrap loadFn with delay logic for SSR but it looks dirty.
Complete example of our work-in-progress loader is
// lazy.ts
import baseLoadable, { DefaultComponent, LoadableComponent, Options } from '@loadable/component';

const LAZY_DEFERRED_SLEEP = 50;
const LAZY_DEFERRED_DELAY = 500;

type LazyOptions = {
    sleep?: number;
    delay?: number;
};

declare function loadableFunc<T>(
    loadFn: (props: T) => Promise<DefaultComponent<T>>,
    options?: Options<T> & LazyOptions
): LoadableComponent<T>;

type LoadableFunction = typeof loadableFunc;

const isSSR = process.env.REACT_APP_TYPE === 'ssr';

export const loadable: LoadableFunction = isSSR ? baseLoadable : (loadFn, options = {}) => {
    const { sleep = LAZY_DEFERRED_SLEEP, delay = LAZY_DEFERRED_DELAY } = options;

    type LoadFn = typeof loadFn;

    let loader = loadFn;

    if (sleep) {
        let sleepCallback = loader;

        loader = props => new Promise(resolve =>
            setTimeout(() => resolve(sleepCallback(props)), sleep));
    }

    if (delay) {
        let delayCallback = loader;

        loader = props => Promise.all<ReturnType<LoadFn>, void>([
            delayCallback(props),
            new Promise<void>(resolve => setTimeout(resolve, delay))
        ]).then(([module]) => module);
    }

    return baseLoadable(loader, options);
};

And usage is
// Component.tsx
import { loadable } from '../../helpers/lazy';

const Component = loadable(() => import(
    /* webpackChunkName: "details-screen", webpackPrefetch: true */
    './Component'
));

If no process.env.REACT_APP_TYPE=ssr set we getting error loadable: SSR requires @loadable/babel-plugin, please install it in SSR execution.
So the question is: how to correct wrap import statement for @loadable/component in SSR environment?
Sorry for my English.
Have a nice day!

Hello Dmitry, long time no see. The wind map is just 👍

sleep and delay in your case are doing the same thing - delaying promise resolution. While sleep can defer script loading, delay will not defer spinner display. If you need to defer it(especially if you use Suspense) - move that "delay" to the Spinner itself.

loadable: SSR requires @loadable/babel-plugin, please install it

Is a very common issue. And the simple - https://github.com/gregberge/loadable-components/blob/6ea00c4c2bdaaf6c603def69ea45c57a6a0d01c9/packages/component/src/createLoadable.js#L59 - basically babel-plugin didn't do the job.
In short - it transforms any loadable, as explained here, and the code is straight forward - if you call loadable - transform!
Something got broken in a few lines.
What could go wrong

it's not "loadable" (babel rewrote imports to CJS)
there is no import inside (babel did node-import)
that's all

And everything should never happen, as long as babel-plugin is set to perform transformation before any other operation, especially node-import.
However, try to (edit it directly in node_modules) add some debug here and come back with the results.

Hello Anton, nice to see you again! Thanks but its time to make remaster of these maps :)
In our tests mentioned sleep and delay options do expected job. Maybe I just broke the code with ssr edits, its fine at this stage)
Yep Im currently see the problem in code transform stage too and trying to modify my code to match test cases but not tried to deep dive into plugin logic yet. Also it looks like our CreateReactApp-based webpack config made incompatible work before babel-plugin come into.
Thanks for the detailed answer!


TFKUBE-346: Fixed the e2e test by apply the sub-path in dataset script

Started https://github.com/atlassian-labs/data-center-terraform/actions/runs/1890494691
There is also a failing unit tests check

Started https://github.com/atlassian-labs/data-center-terraform/actions/runs/1890494691 There is also a failing unit tests check

The e2e failure is introduced by TFKUBE-281: Bitbucket e2e tests (#169)


Eliminate unedited submissions of applications - perceived duplicates

Is your feature request related to a problem? Please describe.
Program Admins receive duplicate applications (we're not sure why residents submit multiple applications, but it's a big slow down across all programs). It would help if there was a way to reconcile the applications; i.e. decide which one to keep and which to archive/delete so that Admins and Residents are only communicating around 1 application.
Describe the solution you'd like
A way for Program Admins to select the duplicate applications and then identify which one to keep and which to archive/delete.
Additional context
Consider a way to use the status feature to send the applicant an email that an application appears to be a duplicate and was archived.
Done when
What conditions need to be met before this issue can be closed?

Setting this as a P3.  We have initiatives to prevent duplicates which may be where we need to focus our efforts.  Reconciling duplicates may be a task best left to the CRM software (or spreadsheet) used by the receiving department.

This part of https://github.com/civiform/civiform/issues/4022
Duplicates are a result of CiviForm allowing an applicant or TI the ability to start an edit and submit it without changes causing what appears to be a duplicate.  Proposed in PRD that this no longer be allowed to occur or not display these type of edits to be displayed to the program admin.  https://docs.google.com/document/d/1WgrjiPNc27ZcimyqGeCjaFCeScaPnx-e7rEIYk9GylY/edit?usp=drive_link
@sijiayam is working on mocks for a redesign of the application display for Program admins and this should be included in that redesign effor.

As a part of this work is a requirement from Freshbucks to be able to not allow application edits of any kind and to direct the applicant to contact program staff.

Closing to be tracked with https://github.com/civiform/civiform/issues/4869 where a duplicate issue was captured and work is in progress


Feature: serialize EditorState to HTML in read-only mode

I'm storing EditorState as the value for a RichTextInput (controlled input), and I want on form submission to:

:white_check_mark: generate JSON from EditorState -- JSON.stringify does it
:x: generate HTML from EditorState and EditorConfig (styles)  -- I need an editor instance

Currently, to generate HTML from Lexical, I need to use Editor. update  with $generateHtmlfromNodes, which has some issues:

Not possible in read-only mode,
Generating HTML from the editor requires the caller of the method to have an Editor instance, where as an EditorState and EditorConfig should be sufficient

Pleasure to contribute code
I'm open to contributing to this project if this feature is a valid use case.

Editor is needed due to the fact HTML needs the EditorConfig to apply theming. See https://github.com/facebook/lexical/issues/2586 for some background and why it had to be closed.
You can always create a detached editor to do the rendering, or a headless editor and do the work on the server. Or simply store the HTML in state along with the editor state when you save the editor state.
The reason you can’t use the helper function in read only mode is exactly the same problem as the first above. In read only mode, Lexical doesn’t know if any active editor. It’s an editor to get the theme config to apply to the HTML elements.

Yes, I understood that part @trueadm. But as you said we only need EditorConfig, not an Editor or an active Editor.
My use case is to generate HTML at form submission outside the editor component (don't want to pass the editor instance to the outer scope).  Generating HTML every time is overkill.
If the $generateHtmlFromNodes requires an Editor just to get theme config, why not accept directly an EditorConfig as a parameter?
A LexicalNode has createDom(EditorConfig) which means, a node needs just EditorConfig to generate its DOM representation.

@med8bra Well unfortunately, the exportDOM method also requires passing of editor too, as the editor is used to initialize nested editor state – for example image captions. So we making such a big breaking change would still require an editor to be available.

@trueadm Indeed, exportDOM is taking an EditorInstance,   but I just did a search in the repository and I cannot find a case where the editor was useful to generating DOM, except:

LexicalNode#exportDOM: it's using the editor to get element node and extract Excalidraw SVG content.

Sorry, I didn't understand the use case for image captions, won't an ImageNode have the image caption in its state?

My FIndings

Passing an Editor instance could be helpful to optimize DOM generation (just cloning existing element node)

:white_check_mark: delivering fast performance
:x: exporting DOM requires the editor to have a root element, and to have nodes rendered already; which is a bit expensive for some use cases (headless mode)

Exporting DOM by library design should only require EditorState and EditorConfig  (most of the code base respect that)

:white_check_mark: exportDOM contract will be clear, with no side effects, taking two inputs (state and config) and returning the DOM element.
:x: Maybe not fast for some use cases, (e.g. Excalidraw node)

Proposed solution
To benefit from the two strategies, maybe we should define two different use cases:

[FASTEST] Export current editor to DOM: uses current rendered elements and optimizes on that
[ROBUST]  Export editor state to DOM: each node uses its state (from editor state) and editor config to create its DOM representation.

Let me know what is your take on this

One workaround could be to expose a new core helper from lexical called $getEditor() which returns null | EditorInstance. Then Excalidraw could use that instead and we wouldn’t need to pass it to the html function? Did you want to take a look at making that change in a PR?

That would be a good workaround for now, The only issue I see is that if $getEditor returns null, we will be putting more responsibility on LexicalNode.exportDOM, as it should handle both cases.
Actually, I just read the ExcalidrawNode implementation, it's not using the rendered canvas, but using an utility method to convert Excalidraw data to SVG. So it will be the same performance cost without an editor instance.
But I was thinking maybe we could have different modes:

exporting state to dom: implemented in exportDOM (gets only editor state and config)

for Excalidraw, we will use exportToSvg from @excalidraw/excalidraw

exporting editor to dom: implemented in exportCurrentDOM (takes Editor, current rendered element, and target document)

Node could reuse rendered element to optimize exportion
Default implementation is to clone current rendered element; using Document.importNode

I don't want to rush into writing a PR, but if we come up with a better approach I'll be pleased to implement it.
Thanks for your time @trueadm

What's the status on this? Have you started a PR, @med8bra?
I'd like to run "$generateHtmlFromNodes(editor, null)" (inside "editor.registerUpdateListener") whenever the user has not typed for N milliseconds, in order to send the content to a server, but the trouble is, I enable read-only mode (to remove red lines from chromes spelling check, etc.) when the user clicks outside the box.
Or is there another way to export the content for syncing with a server while in read-only mode?
Or should I rather do a hackish solution, listening on read-only state change and from there add/remove the other listener?

@norregaarden-scalgo  I'm waiting for a confirmation from @trueadm  to start on my proposition.
For your case, using LexicalOnChangePlugin with a debounced onChange callback will be sufficient.

@norregaarden-scalgo I'm waiting for a confirmation from @trueadm to start on my proposition.
For your case, using LexicalOnChangePlugin with a debounced onChange callback will be sufficient.

I can't seem to find the OnChangePlugin anywhere?

@norregaarden-scalgo I'm waiting for a confirmation from @trueadm to start on my proposition.
For your case, using LexicalOnChangePlugin with a debounced onChange callback will be sufficient.

@med8bra thanks you for your suggestion, but the same error pertains using OnChangePlugin from @lexical/react/LexicalOnChangePlugin, i.e. $generateHtmlFromNodes can only be called from editor.read or editor.update, and if I wrap it in that, it says " Cannot use method in read-only mode." even when I'm, not in read-only mode, if the editor has been in read-only mode at some point.

is there any working temporary workaround to get an HTML string on change with $generateHtmlFromNodes?
When the JSON serialization is not an option it seems a quite basic behaviour in order to save the user entered HTML and let them update it later on.
thanks!

Well you can always just document.querySelector(".editor-container").innerHTML

@kuus Why can't you use $generateHtmlFromNodes. That is the correct way to do what you're asking. You can do it in an update listener or using the OnChangePlugin as mentioned above.

$generateHtmlFromNodes can only be called from editor.read or editor.update, and if I wrap it in that, it says " Cannot use method in read-only mode." even when I'm, not in read-only mode, if the editor has been in read-only mode at some point.

Make sure you call $generateHtmlFromNodes from an editor.update, if you are doing so, please can you share a sandbox with a repro case? It's hard to understand what is going wrong otherwise.

Hi @trueadm , thanks for the quick reply, I was trying to do it from editor.read() as such:
import { useCallback, useEffect } from "react";
import { $getRoot, $getSelection, type EditorState } from "lexical";
import { $generateNodesFromDOM, $generateHtmlFromNodes } from "@lexical/html";
import { RichTextPlugin } from "@lexical/react/LexicalRichTextPlugin";
import { ContentEditable } from "@lexical/react/LexicalContentEditable";
import { OnChangePlugin } from "@lexical/react/LexicalOnChangePlugin";
import { useLexicalComposerContext } from "@lexical/react/LexicalComposerContext";

type HTMLEditorContainerProps = {
  defaultValue?: string;
};

export const HTMLEditorContainer = (props: HTMLEditorContainerProps) => {
  const { defaultValue } = props;
  const [editor] = useLexicalComposerContext();

  const onChange = useCallback(
    (editorState: EditorState) => {
      editorState.read(() => {
        const selection = $getSelection();
        const html = $generateHtmlFromNodes(editor, selection);
        console.log("onChange html", html); // gives error Cannot use method in read-only mode.
      });
    },
    [editor]
  );

  // set defaultValue
  useEffect(() => {
    if (!defaultValue || !editor) {
      return;
    }

    editor.update(() => {
      const parser = new DOMParser();
      const dom = parser.parseFromString(defaultValue, "text/html");
      const nodes = $generateNodesFromDOM(editor, dom);
      $getRoot().select();
      const selection = $getSelection();
      // @ts-expect-error not sure...
      selection.insertNodes(nodes);
    });
  }, [editor]);

  return (
    <>
      <RichTextPlugin
        contentEditable={<ContentEditable />}
        placeholder="Placeholder..."
      />
      <OnChangePlugin
        onChange={onChange}
        ignoreHistoryMergeTagChange
        ignoreInitialChange
        ignoreSelectionChange
      />
    </>
  );
};

and that was giving me the error Cannot use method in read-only mode.
If I do it from editor.update it works instead:
import { useEffect } from "react";
import { $getRoot, $getSelection } from "lexical";
import { $generateNodesFromDOM, $generateHtmlFromNodes } from "@lexical/html";
import { RichTextPlugin } from "@lexical/react/LexicalRichTextPlugin";
import { ContentEditable } from "@lexical/react/LexicalContentEditable";
import { useLexicalComposerContext } from "@lexical/react/LexicalComposerContext";

type HTMLEditorContainerProps = {
  defaultValue?: string;
};

export const HTMLEditorContainer = (props: HTMLEditorContainerProps) => {
  const { defaultValue } = props;
  const [editor] = useLexicalComposerContext();

  // set defaultValue and register onChange
  useEffect(() => {
    if (!defaultValue || !editor) {
      return;
    }

    editor.registerUpdateListener(() => {
      editor.update(() => {
        const html = $generateHtmlFromNodes(editor, null);
        console.log("onChange html", html);
      })
    });

    editor.update(() => {
      const parser = new DOMParser();
      const dom = parser.parseFromString(defaultValue, "text/html");
      const nodes = $generateNodesFromDOM(editor, dom);
      $getRoot().select();
      const selection = $getSelection();
      // @ts-expect-error not sure...
      selection.insertNodes(nodes);
    });
  }, [editor]);

  return (
    <>
      <RichTextPlugin
        contentEditable={<ContentEditable />}
        placeholder="Placeholder..."
      />
    </>
  );
};

is this latter the right way to do it?

a bit unrelated (if you like I can open an issue or ask on SO), is it possible to strip the classNames on the resulted HTML string when using $generateNodesFromDOM @trueadm ?

@kuus There's no built in way to do so, your best bet is to apply a query selector on all the nodes and manually do element.removeAttribute('class'). As for using editor.update, that should be fine :)

That's a neat trick too!

@trueadm
I'm storing the json state. I'd like to parse this stored state and send an email (as HTML) on the server?
Right now - my best bet is to use a NodeJS instance on the server to transform JSON to HTML using $generateNodesFromDOM.
There seems to be no way to generate HTML from JSON with zero dependecies. Is this a valid use case for lexical to support?

Generating HTML is contextual for now, so it needs an editor instance to get the node classes from. There’s no way around that

Just to add to this, we still require an editor instance, but the fundamental issue of converting editor state to HTML in non-editable mode has been fixed. You can call editor.getEditorState().read(() => $generateHtmlFromNodes(editor, null)) with editable: false and it should work fine - no need to call editor.update any more.

Closing this as the original bug has been fixed, and we don't have any immediate plans to separate the editor instance from converting JSON state to HTML. That would require rewriting a lot of internal editor logic.

@kuus There's no built in way to do so, your best bet is to apply a query selector on all the nodes and manually do element.removeAttribute('class'). As for using editor.update, that should be fine :)

@trueadm For now I was able to accomplish that by using a kind of dummy headless editor whose state is in sync with the "real" one, the dummy editor has no theme applied to it so no html classes and the onChange is bound to that. It works, but do you think is that a good approach?

Are you able to use html classes from the original editor it is bound to, with that setup?

@kuus There's no built in way to do so, your best bet is to apply a query selector on all the nodes and manually do element.removeAttribute('class'). As for using editor.update, that should be fine :)

@trueadm For now I was able to accomplish that by using a kind of dummy headless editor whose state is in sync with the "real" one, the dummy editor has no theme applied to it so no html classes and the onChange is bound to that. It works, but do you think is that a good approach?

Are you able to use html classes from the original editor it is bound to, with that setup?

hi @erikmartinessanches yes html classes are used in the editor shown to the user. I was looking to strip them out from what saved to the database in the "dummy headless editor" because I only needed to save basic html tags like b,i,em,strong, etc., so there was no need to store html classes too in the database.


Replace deprecated FindBugs annotation with a recommended one

We can now use the new FindBugs annotation SuppressFBWarnings in place of the deprecated SuppressWarnings, whose name overlaps with the Java-standard SuppressWarnings.

lgtm


discussion: what would it take to make this work with enterprise github?

This would be really useful in an enterprise environment like mine where we have our own enterprise instance of github
what would it take to make this work with enterprise github?

If the remote URL can be used to derive the GHE URL, we could generalize the code here: https://github.com/Microsoft/vscode-github-issues-prs/blob/master/src/github-issues-prs.ts#L358
We use the GitHub NPM module which supports GHE as the example in its README shows: https://github.com/mikedeboer/node-github
Not sure what a good strategy is to decide whether or not to try to connect to a URL as GHE. Maybe ask the user the first time.

@mikehaas763 Do you have access to a GHE install? @Ikuyadeu is working on this in #27 and would need some help with testing it.

@chrmarti Yes I do at work

@mikehaas763 I think #27 is ready for testing, could you give it a try?

@chrmarti yes I'll try it out right now

@mikehaas763 I have published @Ikuyadeu's change with the setting hidden. Could you update the extension to the latest version 0.8.0 and set "github.host" to your GHE host for testing? (You can ignore the settings editor complaining about the setting being unknown.)

worked for me - using fqdn
thx!

@chrmarti it worked for me as well!

Republished with the setting revealed. Thanks @Ikuyadeu for the PR!


File Creation - Merge Files

A utility function should exist that takes an array of ACH.Files and merges them into one or many files. If the merged file exceeds the NACHA record limit limitation of 10k entries and the additional file should be created.

// MergeFiles is a helper function for consolidating an array of ACH Files into as few files
// as possible. This is useful for optimizing cost and network efficiency.
//
// Per NACHA rules files must remain under 10,000 lines (when rendered in their ASCII encoding)
//
// File Batches can only be merged if they are unique and being routed to and from the same ABA routing numbers.
func MergeFiles(files []*File) ([]*File, error) { ... }

@wadearnold and/or @bkmoovio would you have other comments to add here?
I'm going to use some of paygate's mergeTransfer logic and line count wrapper for this.

I think it’s good.  Wade can probably chime in on if any specific additional requirements are needed.  It’s great if you can leverage what you did in paygate.


added third_party/sqllite include directory where needed (#205)

fixes #205

Thank you for your contribution! Looks great


Regarding the input of Relation Feature Extractor for CFA

Hello.
I tried to run the python execution in the guidelines in your readme as follows:
CUDA_VISIBLE_DEVICES=0,1 python -m torch.distributed.launch --master_port 10054 --nproc_per_node=2 tools/relation_train_net.py --config-file "configs/e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.USE_GT_BOX True MODEL.ROI_RELATION_HEAD.USE_GT_OBJECT_LABEL False MODEL.ROI_RELATION_HEAD.PREDICTOR MotifPredictor SOLVER.IMS_PER_BATCH 12 TEST.IMS_PER_BATCH 2 DTYPE "float16" SOLVER.MAX_ITER 50000 SOLVER.VAL_PERIOD 2000 SOLVER.CHECKPOINT_PERIOD 2000 GLOVE_DIR glove MODEL.PRETRAINED_DETECTOR_CKPT checkpoints/pretrained_faster_rcnn/model_final.pth OUTPUT_DIR ./exp/motifs_cfa_sgcls TYPE cfa MIXUP.FEAT_PATH feats ``MIXUP.MIXUP_BG`` True ``MIXUP.MIXUP_FG True`` MIXUP.BG_LAMBDA 0.5 MIXUP.FG_LAMBDA 0.5 MIXUP.PREDICATE_LOSS_TYPE MIXUP_CE MIXUP.MIXUP_ADD_TAIL True FG_TAIL True FG_BODY True BG_TAIL True CL_TAIL True USE_PREDCLS_FEATURE False CONTRA True PKO False

However, I found that the feature extractor, AugBilvlMxiUpRelationFeatureExtractor, cannot get the MIX_UP_FG and MIX_UP_BG from the above command. I think the default of mix_up_fg and mix_up_bg is False (the forward function of AugBilvlMxiUpRelationFeatureExtractor), and your code does not put the cfg.MIX_UP_FG itself in the forward function of the relation feature extractor. So, when I run the code, the mix_up_fg and mix_up_bg are False although above command set these variables as True
I think it needs to be corrected. are there things that I misunderstood?

Sorry. I misunderstood the code. I saw the old_roi_features, so I closed my issue.

In the Line 132 in relation_head.py, we set mix_up_fg and mix_up_bg to False to get the original roi features of the objects, which is utilized for the contrastive loss.  It is because that we just want to get original roi features of the objects. The cfg.MIX_UP_FG and cfg.MIX_UP_BG is utilized in Line 135 in relation_head.py to perform feature augmentation. So, there is no error in the code. If you have any other questions, please continue to contact me ^_^.


add -remote flag for remote download

Add -remote for remote download.
Because there is now no https support, some user might want to use scp instead http download.

Many thanks for this @FrontMage, I will review it soon.

Hi @FrontMage, I'm not sure I understand the reasoning behind scp: how does that improve the user experience?
Thanks!

@claudiodangelis Currently there is no support for https, I thought some users might want to use scp instead http download.

@claudiodangelis  Hi is there any feedback on this pr? Since it's been quite a while.

Hi @FrontMage, yeah, sorry for being unresponsive. I have two major comments:

I think we should drop the -ssh flag and related feature, because it's out of scope: setting up the usage of scp from a mobile device so that when the transfer starts through ssh after scanning the QR code adds so much complexity to the process that I guess nobody will ever want to do that. Furthermore, if the purpose of printing out the scp command is letting the user type that command and download the file then I think that a QR utility should not be involved at all

I would change -remote to -external, but I would put this feature on hold until we add HTTPS support

@claudiodangelis Fair enough, after reading about new comments under #24 , I implemented the -cert flag for the client to download certification file, and -external for remote download.

Random thought: since there are already code conflicts (this pull request started a long time ago), wouldn't be better to merge from master first, and then apply changes? Conflicts are major. I can help you with this if you want.

I have another idea: since there are significant conflicts with master, and since this PR has two different purposes (remote/exernal + https support), let's close it and open two new PRs, one for the https support (which should be merged first) and one for the external IP support (which should be taken care of after the first one).
What do you say?

@claudiodangelis Conflicts in main.go is mostly caused by -external flag, I think it's not wise to just put all these in main.go.
I'm thinking that main.go should be a flag parser and command executor, all that http https listen and serve should be in another function in another file.
By doing so, we can upgrade each function separately and leave room for further upgrades.

I'm thinking that main.go should be a flag parser and command executor, all that http https listen and serve should be in another function in another file.

Exactly, totally agree with you @FrontMage.
Let's split the conversation:

HTTPS: https://github.com/claudiodangelis/qr-filetransfer/issues/24
-external flag: https://github.com/claudiodangelis/qr-filetransfer/issues/68

I will close this PR now.


Question: How to strip _(artist) from %search% tag?

Sometimes artist names are like common tags, so many sources use the suffix _(artist) to distinguish that.
Since I save everything by artist on my harddrive and I know for myself that the name at the top level is meant to be an artist, I don't need _(artist) in the folder or the filename.
I guess javascript is the key, but I don't know how... therefore I humbly request a spoon with the solution to be jabbed into my face.

I'm not sure I understand.
You say you save by artist, so why do you want to strip this from the search token? Shouldn't you save it using %artist% and want to strip the artist token then?
As for JavaScript, it should be something like (at the beginning of the filename):
if (artist.endsWith(" (artist)")) {
    artist = artist.substr(0, artist.length - 9);
}

I just want to remove "_(artist)" from a search if it's there, yes. Thanks.
Imagine if an artist was named "green_hair", which is also a tag for, well, "green hair", then the tag for the artist would be "green_hair_(artist)"
Since I know that the top level folder would be an artist, I don't need "_(artist)".
Stripping it from the artist tag itself isn't reliable since sometimes the tag resolves to "multiple artists", but still connected to the artist I search for.
That's why I make the artists my seach term, as well as the top level folder for all the images of the artist.
Editing your solution for search gave me a segfault,  (valid filename, though) but that also happens for artist, but I'm not sure if that's not due to something I did. I'll look into that later, but yeah thanks so far.

The segfaults happened due to an empty "conditional filename" setting related to question which I forgot about.


Improve documentation including incorrect Kryo ser. framework docs

Improve documentation including incorrect serialization framework issue described in STORM-566.

Looks good. +1.
And we may add "[STORM-566]" to the beginning of the pull request's title.

Looks good to me +1

+1


Issue with metabat2

Hello, I am attempting to run the test data, and it fails at the metabat2 stage, with the following message:
nanophase meta -l mnt/databases/db/fastq/SRR17913199.fastq -t 24 -o nanophase-out
[2024-10-10 16:13:01] INFO: nanophase (meta) starts
[2024-10-10 16:13:01] INFO: Command line: /miniconda/envs/nanophase/bin/nanophase meta -l mnt/databases/db/fastq/SRR17913199.fastq -t 24 -o nanophase-out
[2024-10-10 16:13:01] INFO: long_read_only model was selected, only Nanopore long reads will be used
[2024-10-10 16:13:01] CHECK: Nanopore long-read (fastq) file has been found
[2024-10-10 16:13:01] CHECK: Check software availability and locations
[2024-10-10 16:13:01] INFO: The following packages have been found
#package             location
nanophase            /miniconda/envs/nanophase/bin/nanophase
flye                 /miniconda/envs/nanophase/bin/flye
metabat2             /miniconda/envs/nanophase/bin/metabat2
maxbin2              /miniconda/envs/nanophase/bin/run_MaxBin.pl
SemiBin              /miniconda/envs/nanophase/bin/SemiBin
metawrap             /miniconda/envs/nanophase/bin/metawrap
checkm               /miniconda/envs/nanophase/bin/checkm
racon                /miniconda/envs/nanophase/bin/racon
medaka               /miniconda/envs/nanophase/bin/medaka
polypolish           /miniconda/envs/nanophase/bin/polypolish
POLCA                /miniconda/envs/nanophase/bin/polca.sh
bwa                  /miniconda/envs/nanophase/bin/bwa
seqtk                /miniconda/envs/nanophase/bin/seqtk
minimap2             /miniconda/envs/nanophase/bin/minimap2
BBMap                /miniconda/envs/nanophase/bin/BBMap
parallel             /miniconda/envs/nanophase/bin/parallel
perl                 /miniconda/envs/nanophase/bin/perl
samtools             /miniconda/envs/nanophase/bin/samtools
gtdbtk               /miniconda/envs/nanophase/bin/gtdbtk
fastANI              /miniconda/envs/nanophase/bin/fastANI
All required packages have been found in the environment. If the above certain packages integrated into nanophase were used in your investigation, please give them credit as well :)
[2024-10-10 16:13:01] INFO: long-read assembly has been found in the folder: nanophase-out/01-LongAssemblies/. Now go to the next stage: generating LongBins...
        Note: please ensure flye assembly finished successfully in the previous run, if not, please remove this folder using the command 'rm -rf nanophase-out/nanophase-out/01-LongAssemblies/' and re-run nanophase command
[2024-10-10 16:13:01] INFO: metabat2 binning re-starts
/miniconda/envs/nanophase/bin/nanophase.meta: line 251: 11348 Segmentation fault      metabat2 -t $N_threads -i $OutDIR/01-LongAssemblies/assembly.fasta -o $OutDIR/02-LongBins/INITIAL_BINNING/metabat2/metabat2-bins/bin -a $OutDIR/02-LongBins/INITIAL_BINNING/metabat2/metabat2_abun.txt --cvExt > $OutDIR/02-LongBins/INITIAL_BINNING/metabat2/bin.log
[2024-10-10 16:13:02] ERROR: Something wrong with metabat2 binning, please also check nanophase-out/02-LongBins/INITIAL_BINNING/metabat2/bin.log, terminating...

The assembly outputs seem fine. I have attempted this with several different values for number of threads, but no success.
Any advice would be greatly appreciated!

Thanks for trying nanophase.
This might be related to a memory issue. Could you provide the memory specifications of the server you are currently running it on, as well as the operating system?

I am attempting to run it in a docker container on a Windows PC. The container is set up as a Linux system, and it has 32GB of RAM.

That might be the issue. You can refer to this link — hopefully, it will help! :)
I'm not sure what type of sample you will work with, but generally, 32GB of memory might not be sufficient for handling complex metagenomic sequencing datasets.


Recipe Catalog entries are dissapearing

I updated the ai plugin on March 4th, and I could use the code gen application. Cloning it worked perfectly, earlier versions would just spin and not seem to do anything. This time it worked great; however, the next day the plugin no longer shows the code gen application. I can see the repo cloned under ~/podman-desktop/codegeneration along w/ the model, but neither can be viewed from the plugin any more.
Sorry I don't know how to report the exact version I'm running or more details.

Hi @mrguitar thanks for reporting this issue!

Could you tell me which version of Podman-Desktop you are using ?
Could you also explain how you installed the AI-Studio ?

If you have some time to replicates, and extract the logs, we would really appreciate, to do so, here are the steps to follow:
(1) Go to settings -> extensions -> AI Studio and check if there is an error in this page

If there is, please copy it and paste it in this issue.
(2) Disable then Enable the extension

This step is to ensure the last logs will contain the potential error.

(3) Go to the troubleshoot page

(4) Select the Logs tab
(5) Copy/paste the content in this issue

I'm on version 1.7.1.
I installed the extension by pasting: ghcr.io/projectatomic/ai-studio:latest into Install a new extension from OCI Image
There isn't any visible text under the extension like in your screenshot.
Here are the logs. I didn't see anything obvious, but hopefully you'll see the issue. After disabling/enabling the plugin, the codegen app still isn't showing in the UI.
podman.log

FYI, switching to the :nightly tag on the plug in brought back the app that disappeared. I believe this is resolved so I'm closing this.


Enable 'Cancel' button of progressive notification

Previously 'Cancel' button of progressive notification
while running onecc showed NYI error.
With this commit, 'Cancel' button actually terminates
onecc progress.
for For https://github.com/Samsung/ONE-vscode/issues/715
draft #716
ONE-vscode-DCO-1.0-Signed-off-by: Hyun Sik Yoon<EMAIL_ADDRESS>

GIF
Please go see at https://github.com/Samsung/ONE-vscode/issues/715

😮 not requested yet..?!

How about "The job is cancelled" instead of "Terminated successfully" as a cancellation message? Because the word 'successfully' may bring about a miscomprehension of "A job terminated successfully", not "A cancellation terminated successfully".

@dayo09
Message text was changed. PTAL.

Thanks a lot! :-D

@llFreetimell mentioned

SelfKilled in the resolve('SelfKilled') may be handled by string, not SelfKilled.

After discussion, I'd like to rewrite this to
"SelfKilled in the resolve('SelfKilled') may be handled by ExitCode which is string, not by SelfKilled."
This is valid point. I think that I should fix this.

replaced by #810


to be able to row bind: need to combine previous columns date and time_UTC into single column BEGIN_GMT_TOWDATE

example from MVCO nutrients package https://github.com/WHOIGit/nes-lter-nutrient/blob/c1ecadd8d4d6ba008f9c0104504b8fdc1d6f75f8/nutrient-mvco-eml-assembly.RMD#L55


Update job jar location

Update to new job jar location
Update value.avro-confluent.schema-registry.url -> value.avro-confluent.url

https://github.com/streamshub/flink-sql/pull/20 changed the job jar location to /opt/streamshub/
The job was also complaining about deprecated value.avro-confluent.schema-registry.url so I went and changed it.

Could you explain more about this?

The job was also complaining about deprecated value.avro-confluent.schema-registry.url so I went and changed it.

The change doesn't have any problem. Just be interested in the change motivation. Thanks.

When running the job prior to the change you see this in the logs multiple times:

2024-09-06 03:23:46,254 WARN  org.apache.flink.configuration.Configuration                 [] - Config uses deprecated configuration key 'value.avro-confluent.schema-registry.url' instead of proper key 'value.avro-confluent.url'


More up to date README from the java version

Too much copy paste - it's .net
Feel free to merge when you've fixed that

@aslakhellesoy doh - thanks :)


The SMART pattern in pattern fingerprints

Hi there,
As mentioned in a submitted issue before, the pattern fingerprint was recommended for screening substructures.  And it seems to me that the structural pattern of each bit in fingerprint should be pre-defined, like MACCS fingerprints. However, I could not find useful information about the exact definition of each bit in pattern fingerprints. So is there a way to understand the SMART pattern corresponding to each bit?
By the way, here is my code to generate pattern fingerprint:
from rdkit import Chem
from rdkit.Chem.rdmolops import PatternFingerprint
smi = 'CCCc1nn(C)c2C(=O)NC(=Nc12)c3cc(ccc3OCC)S(=O)(=O)N4CCN(C)CC4'
m = Chem.MolFromSmiles(smi)
fp = PatternFingerprint(m)
print (fp.ToBitString())

Thanks a lot!

Hi,
The Pattern fingerprint works differently from MACCS. Instead of having a fixed SMARTS for each bit, the Pattern fingerprint has a small number of very generic patterns. When one of these matches the molecule, the specific atoms and bonds it matches are hashed to give a bit ID. There's an added complication that bits are also set for the generic patterns themselves . This last part makes it more or less impossible to provide reliable information about which atoms contribute to a particular bit.
I really should write this up for the documentation.

Okay, I see. Much appreciate for your rapidly answer.
However, sometimes there is a need to generate as many substructures/patterns as possible, and then find out whether these substructures exist in a set of molecules.  I think using fingerprints that are pre-defined for each bit is the fastest way. But MACCS only has 166 bits, which carries little information. Another way I think about it is that, artificially define a number of substructures, and find whether they exist in molecules. But how to define is still a problem. Or maybe there exists a database of all kinds of SMART patterns? Anyway, this way should be time-consuming. So I wonder whether there is a way to implement this function...
Many thanks again! And hope you can share some ideas about this.

I'm not quite sure what you're trying to do, but there are many ways to generate libraries of fragments to search through. Finding interesting ones is a different matter entirely. :-)
Here's one piece from the RDKit documentation that might be helpful in terms of starting to find a collection of fragments: https://www.rdkit.org/docs/GettingStartedInPython.html#molecular-fragments

Got it, thank you!

@greglandrum hello，Could you tell us more about how this PatternFingerprint was generated, thank you very much

Hi, The Pattern fingerprint works differently from MACCS. Instead of having a fixed SMARTS for each bit, the Pattern fingerprint has a small number of very generic patterns. When one of these matches the molecule, the specific atoms and bonds it matches are hashed to give a bit ID. There's an added complication that bits are also set for the generic patterns themselves . This last part makes it more or less impossible to provide reliable information about which atoms contribute to a particular bit. I really should write this up for the documentation.

Have you finish your documentation of PatternFingerprint? It is really hard for me to read the source code, hope to get your documentation.

Hi, The Pattern fingerprint works differently from MACCS. Instead of having a fixed SMARTS for each bit, the Pattern fingerprint has a small number of very generic patterns. When one of these matches the molecule, the specific atoms and bonds it matches are hashed to give a bit ID. There's an added complication that bits are also set for the generic patterns themselves . This last part makes it more or less impossible to provide reliable information about which atoms contribute to a particular bit. I really should write this up for the documentation.

Have you finish your documentation of PatternFingerprint? It is really hard for me to read the source code, hope to get your documentation.

https://www.rdkit.org/docs/RDKit_Book.html#pattern-fingerprints


The Python/Node.js code corresponding to the advanced parameters in the image playground is incorrect

Describe the bug

Some parameters are not compatible with OpenAI

Environment

GPUStack version: main 31fa6d1
OS: Ubuntu 22.04
GPU: NVIDIA 4090

Additional context
Add any other context about the problem here.

UI version: cd1d300


Fix await/yield/yield from in genexpression iteration

Do this by ignoring the generator expression iterator when searching for yield/await nodes - the iterator is evaluated in the outer scope instead.
Fixes #5851

You also need to adapt the .pxd file.

3.0.7 commit: https://github.com/cython/cython/commit/3c4acd2c35ffcfd381ddc2521992ea324831d3cb


rplidar_ros: 2.1.3-2 in 'humble/distribution.yaml' [bloom]

Increasing version of package(s) in repository rplidar_ros to 2.1.3-2:

upstream repository: https://github.com/Slamtec/rplidar_ros
release repository: https://github.com/ros2-gbp/rplidar_ros-release.git
distro file: humble/distribution.yaml
bloom version: 0.11.2
previous version for package: 2.1.0-1

rplidar_ros
* Rename SLLidar* to RPLidar* in rplidar_node.cpp
* Renaming variable m_running to is_scanning
* Update README
* Use rplidar_ros.rviz instead of rplidar.rviz
* Update source files in src directory:
  Use the source files developed by Slamtec instead of the old ones, and add the functionality of the old code.
* Modify and add launch files
* Update rplidar-sdk to 2.0.0
* Update create_udev_rules.sh
* Update description in package.xml
* Update maintainer to Wang DeYou
* Contributors: Wang DeYou

tag error


Problems with the JDK 131 related release

Hello,
I see you're repushing the JDK 131 related images to the Docker Hub. These days we ran into trouble because we were using the slim image as parent image for our projects and their stop recognizing the docker enviroment variables once updated to 131. There was no problem with 121 images, so yesterday we change our code to stick to them. I was preparing a test project to share with you.
Is it a problem you've already solved with the current push?
Thanks

Hi,
I haven't changed much in the images, they have automatically got rebuilt with the latest commit, which only touched README.
I have not experienced any problems with the image in any of my projects. A week or so ago I have updated the base image from Alpine 3.5 to the newly released Alpine 3.6, so it might be the cause of the change in behavior. Can you provide a minimal reproducible example so I can help you?
Cheers!

@xtremebiker: I am facing the same issues in passing --env variables to the docker run command. The springboot container works file if I use frolvlad/alpine-oraclejdk8:8.121.13-slim but it ignores --env variables with frolvlad/alpine-oraclejdk8:8.131.11-slim
Were you able to resolve it?

@salitrapraveen haven't found time still for it. I'm also using Spring Boot and that's the format of my docker files:
Works:

FROM frolvlad/alpine-oraclejdk8:8.121.13-slim
ADD app.jar app.jar
ENTRYPOINT exec java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /app.jar

Doesn't pick ENV variables:

FROM frolvlad/alpine-oraclejdk8:8.131.11-slim
ADD app.jar app.jar
ENTRYPOINT exec java $JAVA_OPTS -Djava.security.egd=file:/dev/./urandom -jar /app.jar

Still, I made some tests in my local machine and couldn't reproduce it in a project made from scratch (Spring Initializr). Could be even a problem with the Spring Boot version (I'm using 1.4.4 for my production projects) and that specific JDK version. It might be interesting testing with 1.4.4 and more recent boot versions and this specific jdk image.

I am sorry, but I cannot reproduce the described behaviour on my end:
FROM frolvlad/alpine-oraclejdk8:8.131.11-slim

ENTRYPOINT exec echo $JAVA_OPTS

$ docker build -t qq .
$ docker run -it --rm -e JAVA_OPTS=java-test qq
java-test


:wrench: fix(type annotation): Add SelectOfScalar to AsyncSession exec statement type annotation

Hi, the statement argument type annotation of the exec method in the AsyncSession doesn't include SelectOfScalar[_T] causing mypy to complain about using await session.exec(select(Model)):
error: Argument 1 to "exec" of "AsyncSession" has incompatible type "SelectOfScalar[Model]"; expected "Union[Select[<nothing>], Executable[<nothing>]]"

Since the synchronous self.sync_session.exec is used under the hood, type annotations for the statement argument should be the same.

Hi, just a question, if you don't mind:
We are using, with SA 1.4, session and select from there directly:
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.future import select

is there some benefit to using them via SQLModel? I haven't noticed any problems with just using the SA ones directly. The version in SQLModel that this PR touches does not seem to do much?

Thank you! This was solved in https://github.com/tiangolo/sqlmodel/pull/58
The fix will be available in the next version, 0.0.9 🎉
Given that I'll close this one, but thanks for the work! ☕


Update documentation to match the new App and Model classes

These pages:

https://docs.tinychain.net/guides/chain-persistent-mutable-state
https://docs.tinychain.net/guides/object-orientation
https://docs.tinychain.net/guides/host-your-service

Have been badly out-of-date since Cluster was replaced by App. We should update them based on the App and Model examples in the tests/apps directory.

Hey haydnv, I may need access to your gitbooks.for the time being if you could review the docs i'll add them to the comments.

Chain: persistent mutable state.
In order to serve a dynamic application, you’ll have to have a way of updating your service’s persistent state. To do this you can use a Chain, a data structure which keeps track of mutations to a Collection in order to maintain the consistency of that State across every replica of an
app.
class Persistent(tc.app.App):
    __uri__ = tc.URI("http://<IP_ADDRESS>:8702/test/chain")

    def _configure(self):
        self.rev = tc.Chain.Sync(0)
        tc.app.App.__init__(self)

    # this method looks up the value of "rev" within a transaction
    @tc.get
    def version(self) -> tc.Number:
        return self.rev

    # this method updates the value of "rev" within a transaction
    @tc.post
    def bump(self, txn):
        # note: the value of "version()" must be assigned a name
        # in order to be addressable by the instance method __add__ below
        txn.rev = self.version()
        return self.rev.set(txn.rev + 1)

When you start a TinyChain host with an App definition, it will assume that there is a hosted app with the given URI (”http://…/app/test/chain” in the example above) and attempt to join that App as a replica.
If the app URI has no host address, or is the same as the address of the running host, the host will serve a single replica of a new app.
Watch out for versioning issues! In production, it’s best to end your app URI with a version number which you can update in order to release a new, non-backwards-compatible version with different data and methods. We are attempting to find a solution to this and will be released at a future time.

Object orentation
One of TinyChain’s most powerful features is its object-oriented API.
You can use this to define your own classes, which must inherit from exactly one class, which ultimately inherits from a native class.
For example:
import tinychain as tc

URI = tc.URI("http://example.com/app/area") # <-- edit this

# Specifying `metaclass=tc.Meta` provides JSON encoding functionality for user-defined classes.
# It's only required when subclassing a native class -- subclasses of `Distance` automatically inherit its metaclass.

class Distance(tc.Number, metaclass=tc.Meta):
    __uri__ = URI.append("Distance")

    @tc.get
    def to_feet(self) -> Feet:
        return tc.error.NotImplemented("abstract")

    @tc.get
    def to_meters(self) -> Meters:
        return tc.error.NotImplemented("abstract")

class Feet(Distance):
    __uri__ = URI.append("Feet")

    @tc.get
    def to_feet(self) -> Feet:
        return self

    @tc.get
    def to_meters(self) -> Meters:
        return self / 3.28

class Meters(Distance):
    __uri__ = URI.append("Meters")

    @tc.get
    def to_feet(self) -> Feet:
        return self * 3.28

    @tc.get
    def to_meters(self) -> Meters:
        return self

TinyChain does not have any concept of member visibility, like a “public” or “private” method.
This is because TinyChain objects are meant to be sent over the network and used by client code, making a “private” method is meaningless (and deceptive to the developer implementing it).
If you want to hide an implementation detail from the public API of your class, consider using a helper function outside your class definition.

host your service (WIP)
Now that we've defined some classes in  and a Chain in , it's time to put our service online! You can do this using the same object-oriented API that you used to build the service. For security reasons, TinyChain requires that you load your Cluster configuration from a local file on the host machine; see the next section,  for instructions.
from __future__ import annotations
import tinychain as tc

HOST = "http://<IP_ADDRESS>:8702"

class Demo(tc.app.App):
    __uri__ = tc.URI(f"{HOST}/demo")

    @tc.post
    def area(self, length: tc.Int, width: tc.Int) -> tc.Number:
        return length * width

def start_host(name, apps):
    from tests.tctest.process import start_local_host, start_docker

    # return start_local_host(name, apps)
    return start_docker(name, apps)

if __name__ == "__main__":
    # start a new TinyChain host using the docker guide.
    start_host("demo", [Demo()])
    host = tc.host.Host(HOST)

    # verify that it works as expected
    resp = host.get("/")
    print(resp)
    assert host.post("/demo/area", {"length": 5, "width": 2}) == 10

You can see more in-depth examples in the tests directory.

With this one I think there is another bug in tiny chain, every time I come to run the code using the latest tinychain bin (0.9.0-beta)
start host process on port 8702
bin/tinychain: error while loading shared libraries: libaf.so.3: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/cormac/workspace/python/tinychain/script.py", line 25, in <module>
    start_host("demo", [Demo()])
  File "/home/cormac/workspace/python/tinychain/script.py", line 19, in start_host
    return start_local_host(name, apps)
  File "/home/cormac/workspace/python/tinychain/tests/tctest/process.py", line 206, in start_local_host
    process.start(wait_time)
  File "/home/cormac/workspace/python/tinychain/tests/tctest/process.py", line 141, in start
    raise RuntimeError(f"TinyChain process crashed on startup")
RuntimeError: TinyChain process crashed on startup

readthedocs.io also has not updated since v0.4

The readthedocs.io issue is now fixed: https://tinychain.readthedocs.io/en/latest/


[REQ] Adding a Motion Blur option when shaders are turned on.

Acknowledgement of preliminary instructions

[x] I have read and acknowledged the preliminary instructions.

What idea do you have?
(This is being asked once again on this Github page because that issue is very old and since then Reshade hasn't existed as a thing for Roblox because of the Anti-Cheat and 64-bit update, I hope this doesn't really cause some issues making this not a valid request.)_
So once I was thinking about getting Roblox Motion Blur without having a 244hz monitor. And since I have been a Bloxstrap user since January of 2023 (about when the anti-cheat was added) I wanted that back because of the anti-cheat removing Reshade as a shader platform for Roblox... And then I just remembered that I used the MotionBlur on Reshade, I wanted it back but this will not work because of the Anti-Cheat and the 64-bit update, and that made me a bit upset over why Roblox just didn't add an Anti-Cheat that would not block any time of Game Shaders that a lot of players wanted as a normal function in Roblox. This is once again why I started using Bloxstrap as my driver for playing Roblox...
Anyway thanks for implementing this feature back into Roblox (if it even gets added)
Thanks, -Matthew (My real name is Matthew but since I am from the Czech it is completely different.)

This isn't possible. I can't do this. Sorry.


docs: update devtools setup with Firefox add-on

https://github.com/ksz-ksz/rxjs-insights/commit/387bc76bbfb25b4c688d674a26b7130d4728a7c4
Update Devtools > Setup to also mention Firefox add-on.

Hey, thanks for the fix!


second epoch weights issues

Hi,
Looks like the weights starting from the second epoch, gives an issue while testing with sample image.
The output is only sequence of start  ...
When i debugged, I see that the "outputs"  tensor array in model.py contains all  "nan"

@sujayr91 I don't understand what you are asking.. What do you mean by 'weights starting from the second epoch'?

Hi yunjey,
The model parameters I am talking are with respect to files which will be saved in model/ during training epochs. First I trained the model for Coco dataset.  While testing sample.py,  if i choose model parameters any where starting from second epoch (for example encoder-2-1000.pkl , and decoder-2-1000.pkl) I get output caption as    .......
I will check your pretrained model also.
Thanks

Oh, I think you should change the code in configuration.py.
I deleted the configuration.py file and add argparse to control the arguments. Please check the code.


Add column arn for the table alicloud_oss_bucket. Closes #146

Integration test logs

Logs
Add passing integration test logs here

Example query results

Results
Add example SQL query results here (please include the input queries as well)

@plz add test and doc wherever applicable


Always move label to end of the line

Currently the label moves away from the line if the last column value is missing

Closing this as a subset of the broader discussion of #33.


Remove extra space

@nachoalonsoportillo : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Parsing of infeasible CBC solution fails

When having a (toy) model like
minimize
  obj: a + b
subject to
  c1: a + b <= 1
  c2: a + b >= 2
binaries
  a b
end

the resulting solution file looks like
Infeasible - objective value 2.00000000
**       0 a                      2                       0
      1 b                      0                       0

As the first line is additionally marked with leading '**' parsing fails, as the total number of fields isn't 4 anymore. Thus, an error is returned ("Incorrect solution format") although it could be parsed by simply ignoring the first field (the status itself should be sufficient to mark the result as being potentially unusable).

Thanks for your fix !


Can we improve search speed through additional pre-processing (potential learning experience?)

If we remove bad peaks from our experimental spectra, then there are fewer peaks that have to be compared to each theoretical. This improves both our confidence as well as our search speed.
Here are a few coding projects that people may (or may not) find interesting, especially with new students coming into the lab:

Remove peaks that do not have a neighbor with a mass difference of an amino acid. Fragment ions from the same series tend to be observed consecutively in a row and not randomly scattered. Validation will need to be done to ensure this does not have a negative impact on the FDR.
Often the same peptide can be selected for fragmentation multiple times in a run. Search the MS/MS scans for these events, and remove peaks that were not conserved between them. Validation will need to be done to ensure this does not have a negative impact on the FDR.
Remove peaks that cannot be generated through the selected ion types. This will remove contaminants and internal fragments. Generate a dictionary that can be read by MM for each ion type (up to a certain mass such as 800) and remove peaks from the spectra if they are not found in the specified dictionaries.

https://link.springer.com/article/10.1016%2Fj.jasms.2005.04.010

MS2 spectra are deconvoluted in #1474, effectively removing isotope peaks


Added ANTLRv4 icon

Fixes #65

hi thanks for the contribution, however icon doesnt fit the style guidelines, i'll close the PR but keep the issue open and eventually do it :)

But... the icon is 100 by 100. Where are the style guidelines?

By the way: I put a lot of effort into editing the icon to match the style of the other icons and simply closing this PR without giving me feedback I can build upon is quite rude IMHO.

never meant to, i just struggle with having PRs open that won't be merged atm so i try to avoid being overwhelmed. it's in the works to add a proper contributing guide with guidelines, but we're also smh working on changing them and the way the extension is (to add more customization for example). i added context on the issue you opened and you're still very welcome to re submit a fitting one

I updated the icon according to your suggestions. I also simplified the source file so it is easier to read.

This is how it looks in use in VS Code:

@ricardoboss I'm not maintainer of this repo, but when comparing to other icons this still doesn't seem to follow the icons style: All lines should have the same weight, and the "A" here seems 3x more bold. The entire goal of this icons pack is to make all icons more uniform and coherent together; so maybe you can try reusing the "A" from the icon at the top right in this screen, and mirror it?

I see what you mean, yeah. I tried to stay true to the original logo, but it may be better to adjust it even more. Let me work on it a bit more

@quentinguidee @prazdevs what do you think?

giga merged v1, and antlr icon will be available ! closing this


Report deprecations for deprecated members from the org.gradle.util package

Those were softly deprecated (@Deprecated annotation) in Gradle 7.0 but did not report the deprecation.
Let's add the deprecation reporting in 8.0, they will be removable in 9.0.

Done in https://github.com/gradle/gradle/pull/21843 for Gradle 7.6


Support Cassandra 4.x.x versions, not just 4.0.x / 3.11.x.

…the webhook_test for errorString validation.

What this PR does:
Fixes webhook_test_suite's error parsing to detect errors correctly when the string should've been valid. Also, modify support regexp for Cassandra 4.x.x versions instead of 4.0.x
Which issue(s) this PR fixes:
Fixes #313
Checklist

[ ] Changes manually tested
[x] Automated Tests added/updated
[ ] Documentation added/updated
[x] CHANGELOG.md updated (not required for documentation PRs)
[x] CLA Signed:  DataStax CLA

Are we able to build 4.1 images with which to test?

To build a Management API image for Cassandra 4.1, you have to:

Build a Cassandra Docker image from trunk, based on the scripts DockerHub uses currently
Build a cassandra-all.jar from current Cassandra trunk
Modify the current Management API Dockerfile for Cassandra to pull in the cassandra-all.jar built in the previous step and build the custom Management API image

Once you have that, then you could test.

Are you expecting >= 4.1.0 to require changes on cass-operator side or management-api side? If latter, then there's no reason to block this. If cass-operator side, then that's different story.


Replace stahnma/epel with puppet/epel

Describe the Bug
The spec_helper_local_acceptance set up relies on the puppet module epel, however we are currently using the original deprecated version stahnma/epel rather than the currently maintained puppet/epel.
Expected Behavior
For the code to be update to use puppet/epel

This module doesn't use epel. I think you intended to open this against the apache module.

Yeh, my bad

https://github.com/puppetlabs/puppetlabs-apache/issues/2244


Add grpc samples

Introduce grpc directory, containing a microservice architecture consisting of an internal grpc server, and a public client/server app.
A run.sh script is provided, to easily deploy this architecture in Code Engine, as well as a README, explaining the ideas behind.

@reggeenr thanks for the review. I addressed all your comments. In addition, I added the following:

A build script for pushing the container images of the client and server
An architecture diagram in the README for this sample app


Release testing: Python example parallel-classifier doesn't have a README

ref: #2583
The Python example parallel-classifier doesn't have a README about what it does or how to run it.

This application was built to demonstrate a correctness regression. It's not meant as a user-runnable example. Moving to testing hierarchy.


What are the differences between CVRF and CSAF?

Create short videos (~ 2 minutes long) explaining the content described in the title of this issue. This issue is tracked in the parent issue #30


Optional inputs

Optional inputs:

Student number  DONE
Tags DONE
Address DONE
Postal code DONE

Compulsory inputs:

Name DONE
Class DONE
Grades DONE
Parent number DONE

Coverage remained the same at 92.691% when pulling c0b6369d1b93a664b5e339bbff8e1d0693b5d450 on Lenaldnwj:Optional_Inputs into 84e0e84149142b9104cba3cfabe0805564559929 on CS2103AUG2017-T16-B2:master.

Coverage increased (+0.02%) to 92.715% when pulling 2407e879fc20c0de05e151752107a08b8f20278d on Lenaldnwj:Optional_Inputs into 84e0e84149142b9104cba3cfabe0805564559929 on CS2103AUG2017-T16-B2:master.

Coverage increased (+0.03%) to 92.723% when pulling f1ab92f269522a8d543b4e3862e7756507c25a74 on Lenaldnwj:Optional_Inputs into 84e0e84149142b9104cba3cfabe0805564559929 on CS2103AUG2017-T16-B2:master.

Coverage increased (+0.03%) to 92.723% when pulling 38bedb2a3126b3cf98060b20ff0f101bb8f06a91 on Lenaldnwj:Optional_Inputs into 0288d1140b191baf3e494e8c7cf0172c93fde189 on CS2103AUG2017-T16-B2:master.


ARROW-17645: [CI] Get conda-integration building again

@kou or @assignUser, how does this look for a fix? I'm not happy about pinning more things but it seems zlib hasn't made a new release yet

Can we apply https://github.com/madler/zlib/commit/ec3df00224d4b396e2ac6586ab5d25f673caa4c2 in https://github.com/madler/zlib/commit/ec3df00224d4b396e2ac6586ab5d25f673caa4c2 instead?

I'll try opening a PR and seeing if there's interest in patching

https://github.com/conda-forge/zlib-feedstock/pull/66

Patch was accepted, so once I see the new packages go out I'll test unpinning the dependency

Hmm, it failed, but I see it didn't install the most updated zlib…trying again

Looks like a dummy change to bump the cache did the trick - what's the 'right' way to accomplish that?

I'll remove the cached images listed at https://hub.docker.com/r/apache/arrow-dev/tags (admin page: https://hub.docker.com/repository/docker/apache/arrow-dev/tags ).

I've removed amd64-conda and amd64-conda-cpp.

Thanks! Reverted the last commit, let's see how the build goes now


postprocessing state of a non accessible file should not affect a DELETE

in the https://github.com/owncloud/ocis/pull/9236 the [ci failed](https://drone.owncloud.com/owncloud/ocis/35340/44/6
  @issue-3561 @provisioning_api-app-required
  Scenario Outline: listing other user's trashbin is prohibited for newly recreated user with same name # /drone/src/tests/acceptance/features/coreApiTrashbin/trashbinFilesFolders.feature:211
    Given using <dav-path-version> DAV path                                                             # FeatureContext::usingOldOrNewDavPath()
    And user "testtrashbin102" has been created with default attributes and without skeleton files      # FeatureContext::userHasBeenCreatedWithDefaultAttributes()
    And user "testtrashbin102" has uploaded file "filesForUpload/textfile.txt" to "/textfile0.txt"      # FeatureContext::userHasUploadedAFileTo()
    And user "testtrashbin102" has uploaded file "filesForUpload/textfile.txt" to "/textfile2.txt"      # FeatureContext::userHasUploadedAFileTo()
    And user "Brian" has been created with default attributes and without skeleton files                # FeatureContext::userHasBeenCreatedWithDefaultAttributes()
    And user "testtrashbin102" has deleted file "/textfile0.txt"                                        # FeatureContext::userHasDeletedResource()
    And user "testtrashbin102" has deleted file "/textfile2.txt"                                        # FeatureContext::userHasDeletedResource()
    And user "testtrashbin102" has been deleted                                                         # FeatureContext::userHasBeenDeleted()
    And user "testtrashbin102" has been created with default attributes and without skeleton files      # FeatureContext::userHasBeenCreatedWithDefaultAttributes()
    And user "testtrashbin102" has uploaded file "filesForUpload/textfile.txt" to "/textfile3.txt"      # FeatureContext::userHasUploadedAFileTo()
    And user "testtrashbin102" has deleted file "/textfile3.txt"                                        # FeatureContext::userHasDeletedResource()
    When user "Brian" tries to list the trashbin content for user "testtrashbin102"                     # TrashbinContext::userTriesToListTheTrashbinContentForUser()
    Then the HTTP status code should be "404"                                                           # FeatureContext::thenTheHTTPStatusCodeShouldBe()
    And the last webdav response should not contain the following elements                              # TrashbinContext::theLastWebdavResponseShouldNotContainFollowingElements()
      | path          | user            |
      | textfile0.txt | testtrashbin102 |
      | textfile2.txt | testtrashbin102 |
      | textfile3.txt | testtrashbin102 |

    Examples:
      | dav-path-version |
      | new              |
      | spaces           |
        Failed step: And user "testtrashbin102" has deleted file "/textfile3.txt"
        HTTP status code was not 204 while trying to delete resource '/textfile3.txt' for user 'testtrashbin102'
        Failed asserting that an array contains 503.

with a 503 error
	_______________________________________________________________________

		==> REQUEST

		DELETE /remote.php/dav/spaces/48dc4de9-4966-4e45-ad55-7a27e2a7434f$17844abc-e5ad-4797-982b-8d3a0a3fbe61/textfile3.txt
		X-Request-ID: coreApiTrashbin/trashbinFilesFolders.feature:234-222

		<== RESPONSE

		503 Service Unavailable
		X-Xss-Protection: 1; mode=block
		<== RES BODY
		<?xml version="1.0" encoding="UTF-8"?>
		<d:error xmlns:d="DAV" xmlns:s="http://sabredav.org/ns"><s:exception>Sabre\DAV\Exception\ServiceUnavailable</s:exception><s:message>internal error: file is processing</s:message></d:error>

A file is uploaded and immediately deleted:
    And user "testtrashbin102" has uploaded file "filesForUpload/textfile.txt" to "/textfile3.txt"      # FeatureContext::userHasUploadedAFileTo()
    And user "testtrashbin102" has deleted file "/textfile3.txt"                                        # FeatureContext::userHasDeletedResource()

We should not die with a 503 here ... and this seems to be racy.
IMO it should be possible to delete a file even if postprocessing has not finished, yet.

Yes, it would be nice if the delete is "queued" to be done when the file post-processing is done. (Or a way to "abort" the post-processing and the file never "really exists"). And the DELETE request returns a success status.
Otherwise, the DELETE request could return a 425 to indicate that the post-processing is not yet complete - and it is the responsibility of the client to try again after "some time". That is what happens for a GET (download) request.

Hi,
Can I work on this issue ?


Parquet files containing BYTE_ARRAY column fields written by Benthos not readable by either pyarrow or fastparquet

This bug was raised from benthos project.
Using Benthos, I have two tables, one named "ID" which only contains columns containing INT64, and "person", which contains strings as well as INT64s.
What the code looks like:
Using benthos to extract information from "person" table looks like this,
input:
  sql_select:
    driver: postgres
    dsn: postgres://username:password@localhost:5432/default_database?sslmode=disable
    table: person
    columns: [ '*' ]
buffer:
  none: {}
output:
  aws_s3:
    bucket: < BUCKET NAME HERE >
    region: < REGION HERE >
    credentials:
      profile: < PROFILE HERE >
    path: 'stuff/${! timestamp_unix() }-${! uuid_v4() }.parquet'
    batching:
      count: 1000
      period: 10s
      processors:
        - parquet_encode:
            schema:
              - name: "name"
                type: BYTE_ARRAY
              - name: "age"
                type: INT64
              - name: "lastupdated"
                type: BYTE_ARRAY
            default_compression: zstd

and attempting to load the resulting parquet file written to s3 prints this:
import pandas as pd
test = pd.read_parquet('not_working.parquet', engine='pyarrow')
print(test)

->

Traceback (most recent call last):
  File "/Users/tyler/PycharmProjects/sieve_s3/sort_me.py", line 2, in <module>
    test = pd.read_parquet('not_working.parquet', engine='pyarrow')
  File "/Users/tyler/PycharmProjects/sieve_s3/venv/lib/python3.8/site-packages/pandas/io/parquet.py", line 493, in read_parquet
    return impl.read(
  File "/Users/tyler/PycharmProjects/sieve_s3/venv/lib/python3.8/site-packages/pandas/io/parquet.py", line 240, in read
    result = self.api.parquet.read_table(
  File "/Users/tyler/PycharmProjects/sieve_s3/venv/lib/python3.8/site-packages/pyarrow/parquet/__init__.py", line 2827, in read_table
    return dataset.read(columns=columns, use_threads=use_threads,
  File "/Users/tyler/PycharmProjects/sieve_s3/venv/lib/python3.8/site-packages/pyarrow/parquet/__init__.py", line 2473, in read
    table = self._dataset.to_table(
  File "pyarrow/_dataset.pyx", line 331, in pyarrow._dataset.Dataset.to_table
  File "pyarrow/_dataset.pyx", line 2577, in pyarrow._dataset.Scanner.to_table
  File "pyarrow/error.pxi", line 144, in pyarrow.lib.pyarrow_internal_check_status
  File "pyarrow/error.pxi", line 115, in pyarrow.lib.check_status
OSError: Not yet implemented: DecodeArrow for DeltaLengthByteArrayDecoder.

Process finished with exit code 1

However, if only done using INT64, everything works flawlessly.
...
    batching:
      count: 1000
      period: 10s
      processors:
        - parquet_encode:
            schema:
              - name: "age"
                type: INT64
              - name: "id"
                type: INT64
            default_compression: zstd
        - catch:
          - log:
              level: ERROR
              message: "Schema validation failed due to: ${!error()}"
          - bloblang: 'root = deleted()' # Drop messages that fail

import pandas as pd
test = pd.read_parquet('working.parquet', engine='pyarrow')
print(test)

->

   age  id
0   15  10
1   30  20

Process finished with exit code 0

Files mentioned:
parquet_files.zip
Hardware:
MBP 2021 M1

I don't have any educated guesses other than the BYTE_ARRAY conversion from TEXT to bytes isn't being done in UTF-8.

nmind might not be a parquet-go issue


Design plane ticket, summit, airline, hotel, tour package A & tour package B

6/9 done. To be continued tomorrow.


Fix sporadic test failure where user name could be duplicated.

And some other cleanup as well as dropping the timings of the spec from 3.1s to 2.1s
@jrafanie Please review.  Sorry I lumped it all together, but this spec was so :sparkles: I couldn't help myself.  It might help to review commit by commit.
As seen in #5909

:+1:


Prep release

fixes #42
fixes #52
fixes #53
fixes #54
fixes #60
fixes #61
fixes #62
fixes #65
fixes #67

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@b11c963). Click here to learn what that means.
The diff coverage is n/a.

:exclamation: Current head a4ed072 differs from pull request most recent head 5fa280c. Consider uploading reports for the commit 5fa280c to get more accurate results

@@           Coverage Diff           @@
##             main      #56   +/-   ##
=======================================
  Coverage        ?   65.75%
=======================================
  Files           ?       80
  Lines           ?     1130
  Branches        ?        0
=======================================
  Hits            ?      743
  Misses          ?      387
  Partials        ?        0

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more


feat: delete runtime config in module proto

What type of PR is this?

What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?

Additional documentation e.g., design docs, usage docs, etc.:

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

1 of 1 (100.0%)  changed or added relevant line in 1 file are covered.
7 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.03%) to 58.076%

Files with Coverage Reduction
New Missed Lines
%

pkg/cmd/destroy/options.go
7
81.11%

Totals

Change from base Build<PHONE_NUMBER>:
-0.03%

Covered Lines:
7353

Relevant Lines:
12661

💛  - Coveralls


Add HiGHS solver and update solver documentation

This pull request adds the HiGHS solver to the code and updates the solver documentation accordingly. Second try to use HiGHS solver

Error here with accidentally adding jupyter book


fix(tekton): post GA updates

Signed-off-by: Brian Gleeson<EMAIL_ADDRESS>PR summary
The supporting Tekton Pipeline V2 APIs have been udpated after their GA with some minor fixes/changes. Sync the node-sdk to match the updates
These API updates are not likely to be deployed to production until the new year (Jan 2023)
Furthermore, fix up the integration test for tekton pipeline
PR Checklist
Please make sure that your PR fulfills the following requirements:

[x] The commit message follows the Angular Commit Message Guidelines.
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[x] New tests
[ ] Build/CI related changes
[ ] Documentation content changes
[ ] Other (please describe)

Does this PR introduce a breaking change?

[ ] Yes
[x] No

Wont be merging this until the backend API changes are deployed, likely to happen in the new year


[test-triage] chip_sw_data_integrity_escalation

Hierarchy of regression failure
Chip Level
Failure Description
Test chip_sw_data_integrity_escalation has 2 failures.
0.chip_sw_data_integrity_escalation.233406972
Line 1101, in log /container/opentitan-public/scratch/os_regression/chip_earlgrey_asic-sim-vcs/0.chip_sw_data_integrity_escalation/latest/run.log

  UVM_ERROR @ 2657.512088 us: (sw_logger_if.sv:523) [data_integrity_escalation_reset_test_prog_sim_dv(sw/device/tests/sim_dv/data_integrity_escalation_reset_test.c:362)] CHECK-STATUS-fail: %r
  UVM_INFO @ 2657.512088 us: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Steps to Reproduce

GitHub Revision: bc163b70f
dvsim invocation command to reproduce the failure, inclusive of build and run seeds:
./util/dvsim/dvsim.py hw/top_earlgrey/dv/chip_sim_cfg.hjson -i chip_sw_data_integrity_escalation --build-seed 488598207 --fixed-seed 233406972 --waves fsdb

Tests with similar or related failures

[ ] chip_sw_data_integrity
[ ] chip_sw_instruction_integrity
[ ] chip_sw_sram_lc_escalation

Tests failing since May 5th

Tests
2023-05-09 19:59:10
2023-05-09 07:05:17
2023-05-08 22:12:41
2023-05-08 07:09:02
2023-05-07 07:02:50
2023-05-05 07:02:25
2023-05-04 07:05:40
2023-05-03 07:07:45
2023-05-02 07:05:10
Suite

63
chip_sw_data_integrity_escalation
66.67
83.33
50
66.67
66.67
66.67
100
100
100
chip_sw_data_integrity, chip_sw_instruction_integrity, chip_sw_sram_lc_escalation

The failure points to the following section in the test:

This test probably needs to be refactored. All ISRs initiate flash_write transactions, and there is a NMI that also triggers a flash transaction.  When the NMI overrides an ISR in progress, the test fails due to a dif check referenced below.
UVM_INFO @ 2566.742088 us: (sw_logger_if.sv:521) [data_integrity_escalation_reset_test_prog_sim_dv(sw/device/tests/sim_dv/data_integrity_escalation_reset_test.c:325)] At instr access fault handler
UVM_INFO @ 2610.442088 us: (sw_logger_if.sv:521) [data_integrity_escalation_reset_test_prog_sim_dv(sw/device/tests/sim_dv/data_integrity_escalation_reset_test.c:353)] At NMI handler
UVM_ERROR @ 2657.512088 us: (sw_logger_if.sv:523) [data_integrity_escalation_reset_test_prog_sim_dv(sw/device/tests/sim_dv/data_integrity_escalation_reset_test.c:362)] CHECK-STATUS-fail: 0x85861a2e

https://github.com/lowRISC/opentitan/blob/bbebcaff907f3b02206742f6b3b18d0c4a2a40c3/sw/device/lib/dif/dif_flash_ctrl.c#L419-L421

Hi @msfschaffner, I am moving this to M2.5.2 as it will take some effort to remove the flakiness from the test, and it has been root caused to a software issue. Please feel free to move back to M2.5.1 if needed.

That sounds good to me!

Note that there is a new error but also related to this test.
  UVM_ERROR @ 2905.855920 us: (sw_logger_if.sv:523) [data_integrity_escalation_reset_test_prog_sim_dv(sw/device/tests/sim_dv/data_integrity_escalation_reset_test.c:363)] CHECK-STATUS-fail: FLA:209 = Unavailable
  UVM_INFO @ 2905.855920 us: (uvm_report_catcher.svh:705) [UVM/REPORT/CATCHER]
  --- UVM Report catcher Summary ---

Hi @johngt, this is the same error. The difference is that I updated the logging to print the status_t message. This will hopefully make similar issues easier to debug in the future.

We have seen this issue in more recent runs - leaving for @moidx and @msfschaffner to decide whether to close or to put in the backlog as a V3 item. My suggestion would be the latter.

Making this a V3 item sounds good to me!

I think the best way to handle this is to increment nmi_count only once, since that test only cares if the nmi was run at least once. This means only the first NMIs will perform a flash write, so subsequent NMIs won't find the flash to be busy.


Correct CIFAR10_MEAN and CIFAR10_STD in train_cifar.py

For some reason, the CIFAR10_STD used in the example differed significantly from the actual standard deviation of the CIFAR10 train dataset. I corrected both the MEAN and STD with 3 decimal places accuracy.
I calculated the values as seen here: https://gist.github.com/epistoteles/c35bd5154a036748651d8caca11a7efe

Ran with old:
test accuracy: 93.8%
With new/correct mean and std:
test accuracy: 94.0%
Varies a bit over runs of course, but seems fine @andrewilyas

Thanks so much! Merging this now


Note for session IP pinning to work

There was a question raised by a customer on why the session IP address pinning did not work for them after following the current steps in our doc.
After investigating on the issue, it was found out that their org's SSO was configured as optional. For the session IP address to work, it is required that the org must have Required SSO settings configured. This piece of integral information was missing in our docs and hopefully would save us time for debugging when the next customer raise the same issue.

@lizrabuya, I'm skimming through old PRs to see if there are any I can quickly resolve, and this looks like a good candidate. I'll update the branch and merge for you. Let me know if there are any issues with that. 😊


Can't support multiple node service

When trying to add multiple node services , running update-rc.d <app_name> defaults gives:
insserv: script <app_name>: service node already provided! insserv: exiting now! update-rc.d: error: insserv rejected the script header
is there a way to get around this?

Got it, changed the tag Provides: <something>
Great works!