EMFILE: too many open files

Running createImages with a large number of files will result in an error "EMFILE: too many open files".
I was able to work around this by changing the createImages function from
createImages: function (files, callback) {
    // In parallel, calculate each of our images
    async.map(files, this.createImage.bind(this), callback);
}

to
createImages: function (files, callback) {
    // In parallel, calculate each of our images
    async.mapLimit(files, 10, this.createImage.bind(this), callback);
}

How many files were you dealing with?

How many files were you dealing with?

Just over 11000 small images

What's your use case? That seems rather excessive and not something we should support for websites =/ (e.g. resulting spritesheet is still quite large, thus defeating purpose)

I'm working on a game that needs to draw many many sprites and every drawcall to the gpu counts, so it's essential that all the sprites are packed into a single texture. However the individual sprites are very small and being scaled up in the game, so even with 11000 images the actual spritesheet is no larger than 4096x4096.
We use spritesmith to write tools that automate the process of bundling these sprites and relevant data into spritesheets and configurations

Always fascinating to hear how spritesmith transcends it initial use case =) I think the solution to this is:

Use mapLimit with an infinite upper bound (or map if parameter isn't present)
Allow configuration to include some limit functionality

We could also do some try/except backoff looping but this is simpler

Fwiw, EMFILE can sometimes be worked around via ulimit https://stackoverflow.com/a/21201405/1960509 (though you're  even exceeding their number O_O
I'll add support for this by the end of the weekend

Sounds good 👍

Fwiw, EMFILE can sometimes be worked around via ulimit

From what I could find there's no easy way to achieve this on Windows. Maybe with WSL, but i'm not sure it's a good idea to increase the system-wide limit that high :D

We've added options.concurrentFileLimit to the Engine constructor in 2.5.0:
https://github.com/twolfson/pixelsmith/tree/2.5.0#engineoptions
This will automatically propagate to spritesmith and other higher level interfaces (e.g. grunt-spritesmith, gulp.spritesmith) due to floating semver dependencies.
To use this in spritesmith (which it sounds like you're using directly), it should be the params.engineOpts parameter in your constructor:
https://github.com/twolfson/spritesmith/tree/3.4.0#new-spritesmithparams


[HIGH] Give Andrea a higher level access to Checkr than what the info@ Checkr login currently has

In reference to previously posted comment in issue #39, I need password to Checkr admin account so I can have access to Candidate ID's to tack these on to veteran Returning Tutors SDRT User Accounts.
Also, according to Checkr, there seems to be a way for SDRT to change candidate information internally when something goes awry. I am assuming there is a way to do this using the admin account. In this specific case, what is happening is that they used their "preferred name" instead of their "legal name" to start their background check and is now caught in a Checkr Customer Service black hole. Much easier for me to change their name through Checkr using the admin account.
Originally posted by @dreazee in https://github.com/SDRefugeeTutoring/sdrt-custom-functions/issues/39#issuecomment-927985309
@JasonTheAdams @mathetos I'm not sure that adding that field would help. but if that's east to do I could test that out to see if that would be a fix!... An easier fix would be if I can get access to a User's Candidate ID—that would solve everything. And yes, these might be people who have volunteered with us quite a while ago...

@JasonTheAdams I believe this is all set, correct? I've shared with you the credentials in Slack and you should have full access to Checkr.

No, unfortunately. We had a conversation about it on Oct 1, 2021 and it was said that Matt was working to get admin access from Checkr. I had to scroll back in our DM conversations to refresh my memory!

Really? I have both Matt's credentials and the info@ credentials. I remember giving the info@ credentials to you and making sure they have proper access. Is there something else missing?

The info@ credentials is what I have been using this entire time. The functionality on that account is limited. I am unable to remember under what circumstances I have received the "you do not have access to accomplish that task" message, but the next time it happens, I can add that here.

Got it. So I went to update your access... and Matt's password reset again, which means I need his help again. 🤦♂
I'll let you know as soon as I get access again and am able to update your access. Hahah!

@dreazee I got access to Matt's account. He made me an account (to avoid this in the future) and I've updated the info@ account to have admin access.


OS: 12292687: TVS fix for missing lower bound check in enum

@Microsoft/chakra-developers please review


fix crash when continuously clicking back button while drawer is opening

@danghy can you please bump the version up and add a compiled version?

Done @viezel :)

sweet :+1:


Github Actions triggered for every pull request, even if no changes to source are made

For instance builds are triggered for changes to documentation, but documentation doesn't affect build outcome. Can this be made more selective?

My personal experience is that this is a minor problem and opensource project are donated machine cycles for this. It ensures coverage of all the pull requests.

Historically, the godot-python-extensions github actions page says the builds take around a minute. I'm ok with 10 minutes before we need to optimize.
For compatible a regular cached build on godot engine organization is 50 minutes. https://github.com/godotengine/godot/actions/runs/11523267962

Here is the documentation for pull request author requests for ci skip. https://docs.github.com/en/actions/managing-workflow-runs-and-deployments/managing-workflow-runs/skipping-workflow-runs


Markdown EnvironmentVariableCollection.description is not rendered correctly in tooltip

Adopting https://github.com/microsoft/vscode-js-debug/issues/1712, I noticed

/duplicate https://github.com/microsoft/vscode/issues/177774


Kubernetes Mode?

In the maven plugin, it looks like you can specify which mode you are running in.  For the examples i've done, it has always been Openshift mode,  but i do believe there is a straight up Kubernetes mode.
Perhaps we want something like this in the future?

Currently we don't feel there is a compelling technical or business need for this. We can reopen in the future if things change.


Proxy Form only Emails exec + member submitting proxy

See here for details: https://github.com/HSBNE/hsbneportal/issues/92

superseded by #61


ISPN-12398 Update Backup manager resource names and directories

https://issues.redhat.com/browse/ISPN-12398

All addressed, thanks @oraNod

@ryanemerson you need to fix the path in BackupManagerIT

Should be good now

Integrated into master, thanks @ryanemerson !


Report the use of components with vulnerabilities in clusternet

Dear Team Members:
Greetings! Our team is very interested in your project. we performed source code perspective security analysis (SCA) and vulnerability library association analysis on this project and found that components with vulnerabilities are still being used into this project.We would like to report this issue to you,so that you can fix and improve it accordingly. I add the details in json file below. Please confirm whether this problem really exists and confirm with us. Looking forward to hearing from you and discussing more details with us, thank you very much for your time and attention.
Note: Each "affect_components" field in the report represents the vulnerable component introduced by this project. The other is the vulnerability information associated with it.
Qiyu Hou
clusternet-main_report.json

Thanks @HouqiyuA. But we've already got Dependabot alerts #35 and Dependabot alerts #29.


Incomplete Challenge: Invert a Binary Search Tree

Describe your problem and how to reproduce it:
In both the curriculum and guides directories, when you look in the invert-a-binary-tree.md and  index.md respectively, there is no solution and when I go to the challenge as a regular user of fcc, there is no invert method as the challenge description indicates to complete.
Add a Link to the page with the problem:
-https://github.com/freeCodeCamp/freeCodeCamp/blob/master/curriculum/challenges/english/08-coding-interview-prep/data-structures/invert-a-binary-tree.english.md
-https://github.com/freeCodeCamp/freeCodeCamp/blob/master/guide/english/certifications/coding-interview-prep/data-structures/invert-a-binary-tree/index.md
Tell us about your browser and operating system:

Browser Name: Google Chrome
Browser Version: 73.0.3683.103
Operating System: Ubuntu 16.04 LTS

If possible, add a screenshot here (you can drag and drop, png, jpg, gif, etc. in this box):

Hi.  I am wondering if I can work on resolving this challenge e.g. solution possibly, make an invert method etc.  Thank you.

@StanimalTheMan You are correct, those files do not have solutions.  I also checked to see if there were any existing PRs which add solutions to those files and there are none, so you are free to create a PR for the two files.
Thanks for you help.

Alright. Thank you.


Change description

Change the repository's description on GitHub to the one written in the Readme file.

Yep


Build both .Net Framework 4.5 and .Net Framework Core 2.2 project inside same solution

Hello,
I have a project solution that contains both .Net Framework 4.5 and .Net Framework Core 2.2 projects. Previously I have used cake scripts to build other solutions. But for this particular situation, I am getting errors.
Is there any option to do this thing in a single cake script?
Task("Restore-Backend")
    .Does(() =>
    {
        NuGetRestore(solution); //old
        DotNetCoreRestore();
    });

 Task("Build-Backend")
    .Does(() =>
    {
        MSBuild(solution, settings => settings.SetConfiguration(configuration)); //old

         DotNetCoreBuild(rootPath,
             new DotNetCoreBuildSettings()
             {
                 Configuration = configuration,
                 //ArgumentCustomization = args => args.Append("--no-restore"),
             });
    });

What errors are you getting?

if I use DotNetCoreRestore and DotNetCoreBuild the error is as bellow.  All other .Net Core projects are building.
E:\Dipon\Gits\cfs_dy\cfs\CFS\CFS.csproj(720,3): error MSB4019: The imported project "C:\Program Files\dotnet\sdk\2.2.300
\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets" was not found. Confirm that the path in
the  declaration is correct, and that the file exists on disk.
E:\Dipon\Gits\cfs_dy\cfs\CFS\CFS.csproj(720,3): error MSB4019: The imported project "C:\Program Files\dotnet\sdk\2.2.300
\Microsoft\VisualStudio\v16.0\WebApplications\Microsoft.WebApplication.targets" was not found. Confirm that the path in
the  declaration is correct, and that the file exists on disk.

I had a short time, so I had done something like below. It is working form now but still looking for a good solution.
var rootPath = ".";
var solution = rootPath +"/KFS.sln";
var backendProjectAssemblyName = "*.csproj";
var backendProjectPattern = rootPath +"/**/" +backendProjectAssemblyName;

// Run dotnet restore to restore all backend package references.
Task("Restore-Backend")
    .Does(() =>
    {
		NuGetRestore(solution);
        DotNetCoreRestore(rootPath);
    });

// Build c# code using the build configuration specified as an argument.
 Task("Build-Backend")
    .Does(() =>
    {
		List<string> oldFrameworkProjects = new List<string>();
		List<string> coreProjects = new List<string>();
		foreach(var file in GetFiles(backendProjectPattern))
        {
			string version = "";
			version = XmlPeek(file, "/Project[starts-with(@Sdk, 'Microsoft.NET.Sdk')]");
			string path = file.ToString();
			if(string.IsNullOrEmpty(version))
			{
				oldFrameworkProjects.Add(path);
			}
			else
			{
				coreProjects.Add(path);
			}
        }

		/*build .Net Framework projects*/
		foreach(var project in oldFrameworkProjects)
        {
			MSBuild(project, settings => settings.SetConfiguration(configuration));
        }

		/*build .Net Framework Core projects*/
		foreach(var project in coreProjects)
        {
			 DotNetCoreBuild(project,
				 new DotNetCoreBuildSettings()
				 {
					 Configuration = configuration,
					 ArgumentCustomization = args => args.Append("--no-restore"),
				 });
        }
    });

finally, I have done things as below. Working for now
var rootPath = ".";
var solution = rootPath +"/KFS.sln";
var backendProjectAssemblyName = "*.csproj";
var backendProjectPattern = rootPath +"/**/" +backendProjectAssemblyName;

// Run dotnet restore to restore all backend package references.
Task("Restore-Backend")
    .Does(() =>
    {
		NuGetRestore(solution);
        DotNetCoreRestore(rootPath);
    });

// Build c# code using the build configuration specified as an argument.
 Task("Build-Backend")
    .Does(() =>
    {
		List<string> oldFrameworkProjects = new List<string>();
		List<string> coreProjects = new List<string>();
		foreach(var file in GetFiles(backendProjectPattern))
        {
			string version = "";
			version = XmlPeek(file, "/Project[starts-with(@Sdk, 'Microsoft.NET.Sdk')]");
			string path = file.ToString();
			if(string.IsNullOrEmpty(version))
			{
				oldFrameworkProjects.Add(path);
			}
			else
			{
				coreProjects.Add(path);
			}
        }

		/*build .Net Framework projects*/
		foreach(var project in oldFrameworkProjects)
        {
			MSBuild(project, settings => settings.SetConfiguration(configuration));
        }

		/*build .Net Framework Core projects*/
		foreach(var project in coreProjects)
        {
			 DotNetCoreBuild(project,
				 new DotNetCoreBuildSettings()
				 {
					 Configuration = configuration,
					 ArgumentCustomization = args => args.Append("--no-restore"),
				 });
        }
    });

Moving to GitHub Discussions


test: add test for 'unseen' http midstream packets - v1

This comes from an anonymized pcap shared on Discord. While Wireshark
tags the packets as http traffic, Suri only sees tcp, even with
stream.midstream=true.

Does this require a ticket ?

Replaced by: https://github.com/OISF/suricata-verify/pull/877


add missing lodash imports

my project uses inspire-tree-dom but does not use lodash. This PR should make inspire-tree-dom work within my project without having to explicitly require lodash.

fixes #26

I assume so, but these were the only source files missing the imports, correct? I was under the impression there was a lint rule to catch these but apparently not...

I looked through src/, these were the only offenders I found.

thanks for merging! Can you submit a new npm release including this please?

Done, 4.0.6 is out.


Improve Documentation and Sample Code

Thank you very much for making this public. I have been struggling to understand how to fine tune ChatLLaMA as per the provided code snippet in the Readme file. It would be much appreciated if you could clarify the following points in the Readme file.

Are all the training steps (reward, actor and RL training) required (in the order provided in the code snippet) to train ChatLLaMA, or is it sufficient to train the actor only (ActorTrainer)?
The training dataset structures (JSON files) are not clear. In ActorDataset for example, it is mentioned that "completion" was "the output of the user". What does this mean exactly? Does it mean the expected answer to "user_input"? Same for the other training datasets. Examples and more elaborate documentation would be much appreciated.
How to load the ChatLLaMA model weights (from file/directory) for training? Any chance that you could add this to the training code snippet.
It is mentioned in the Readme file that "alternatively, you can generate your own dataset using LangChain's agents". Is this an alternative to the custom dataset? I tried to run generate_dataset.py, but it did not produce any output file, no errors either.

Hello @ylassoued, thank you very much for reaching out.
Re 1: Yes all the steps are required since they reflect the three training steps used for ChatGPT (InstructGPT-like models).

The ActorTrainer fine-tunes the model on your data in a supervised way;
The RewardTrainer trains the reward model in order to simulate the human feedback;
The RLTrainer performs RLHF on the LLaMA model

Re 2: Absolutely, you are right. We are currently working on it in order to refine the code and provide more examples;
Re 3: Sure, we are going to provide a config file ASAP;
Re 4: Yes, it is an alternative to the custom dataset. Thanks for pointing out the issue, I'll investigate the problem.

Thank you very much @diegofiori for your prompt and positive reply! Much appreciated! Looking forward to the next "version" :-).

Hi @ylassoued, examples of the config file and of dataset generation have been added to the new readme with #203 and #204. Feel free to open another issue if you have any suggestion for improving the docs or for getting further information about chatllama 😄

This brilliant!!!! Thank you very much @diegofiori!!! Looks great! I will try give it another try :-)
Thank you very much!


Refactor Cancellables

Unless I'm reading this code incorrectly, there are times when calling cancel() on the cancellableToken will have no effect.

Assigning innerCancellable to a property of a struct should have no effect once cancellableToken is returned to the caller.  The reason it "works" so far, is because performNetworking is usually called synchronously.
The isCancelled property on CancellableWrapper always returns false, so the request would always be sent, even if we fixed (1)

To address these issues:

Make CancellableWrapper a class, so that assigning innerCancellable asynchronously in performNetworking will have the desired effect.
Rename isCancelled to canceled for consistency w/ Cancellable / CancellableToken
Assign a default value to innerCancellable that keeps track of canceled, so that requests can be aborted before they are sent.

          1 Error

:no_entry_sign:
No Changelog entries made

  Generated by :no_entry_sign: danger

These are internal changes, no changelog entry necessary, I think.
If the changes are good, I would like to add specs around cancellables.  Async specs - UG.

Looks 💯 Sorry for radio silence on the other PR, on vacation this week. This looks good to merge when you're ready, I'll review the comments on the other tomorrow.

Some final notes as I let specs run on CI:

Renamed canceled to cancelled, because it's spelled with two LLs throughout the rest of the code.
Prior to this PR, the only place canceled was "exposed" was in CancellableToken as an internal property, so no breaking changes.  In CancellableWrapper it was isCancelled, and this was a private var.
Updated specs accordingly
Added specs to test cancel behavior, and here's the (strange? unintended?) logic that is tested:

if the request is cancelled immediately, the completion handler is not called
if the request is cancelled before the request is sent, the completion handler is not called
if the request is cancelled after the request is sent, the completion handler is called with a .Failure

In most cases, the completion handler will be called, because most people use the default requestClosure, which starts the request before request returns the Cancellable
It's strange, to me, that the completion handler works this way.  Anyone else? CC @Moya/contributors

Executed 102 tests, with 0 failures (0 unexpected) in 3.764 (3.935) seconds
It's danger that is causing circleci to fail 😢
@ashfurrow GTG?

Agreed, that's surprising behaviour. I'll open a separate issue to fix that, this looks great to merge.


Adding an option to force EC2 instance ID as hostname

What does this PR do?
In some cases, users might want to force EC2 instance-id as an hostname.
This is needed when autoscaling a VM with a non standard EC2 hostname
(ie: not starting with "ip-", "domu", "ec2amaz-").
Due to legacy logic GCP and Azure already have the priority when
choosing an hostname but not AWS. On EC2 the OS hostname would be
picked. We need to be backward compatible for the hostname detection so
'ec2_prioritize_instance_id_as_hostname' is introduced.
Describe how to test/QA your changes

Spin a EC2 instance and change the hostname to something else than the default one (ie: something not starting with "ip-", "domu", "ec2amaz-").
Run the Agent and check that the hostname is pulled from the OS (the one you set in step 1.
Now set ec2_prioritize_instance_id_as_hostname to true and check that the hostname is the ec2 instance ID

@hush-hush Just a heads up that the unit test is failing in CI afaict

@hush-hush I feel like this option should be documented in the config template (and mentioned as a changelog entry), any reason not to?


pip install from the source tree does not work

!git clone https://github.com/ImagingDataCommons/idc-index
!cd idc-index && git checkout 0.2.6
!python -m pip install -e ./idc-index

from idc_index import index

client = index.IDCClient()

This results in the import error

I would think it should work?
Demo notebook (I deleted and reconnected to VM between the two experiments): https://colab.research.google.com/drive/1OdLfAAxmSYFJAZ63_L0W6fvVFw2KbiJX?usp=sharing

!git clone https://github.com/ImagingDataCommons/idc-index
!cd idc-index && git checkout 0.2.6
pip install .
get out of the idc-index directory to force importing from python paths rather than local directory..
and then importing and initializing clients worked for me.

This can be closed, following the integration of https://github.com/ImagingDataCommons/idc-index/pull/32
$ mkvirtualenv -p python3.9  test-install-from-source
$ git clone https://github.com/ImagingDataCommons/idc-index.git /tmp/idc-index
$ pip install idc-index
$ python -c "from idc_index import index; client = index.IDCClient()"

Index file not found. Downloading latest version of the index file. This will take a minute or so.
Index file downloaded.

Note that in the example above, the -e is not specified. This is on purpose as creating a editable install would work to disable build isolation as described at https://scikit-build-core.readthedocs.io/en/latest/configuration.html#editable-installs
Last, in the context of notebook and tutorials, I now recommend to simply install the wheel.

While running the test described above in https://github.com/ImagingDataCommons/idc-index/issues/15#issuecomment-1928560437, I also noticed that the output of s5cmd --help was always displayed.
This is now fixed in:

https://github.com/ImagingDataCommons/idc-index/pull/36


Q: How does one setConfigProvider globally?

When one has an Effect, they can change the default config provider like so:
Effect.succeed(42),
Effect.provideLayer(Effect.setConfigProvider(ConfigProvider.fromMap(new Map([["key", "value"]]))))

However, I don't see how I can do this in the context of this plugin. If I add it to the effect resolver, it messes up my types because all the layers need to be provided before I can set a new config provider. I cannot do it in the effect.layers option either for a similar reason, and when I add that to the globalLayer effect options like so:
effectOptions: {
  globalLayer: Layer.provide(
    Effect.setConfigProvider(ConfigProvider.fromMap(new Map([["key", "value"]]))),
    Layer.context()
  ),
},

it type checks but does not seem to work at all. @iamchanii any idea how to configure your own config provider when using this plugin? 🙏

It looks ugly but working.
  builder = new SchemaBuilder<SchemaTypes>({
    effectOptions: {
      globalLayer: Layer.provide(
        Effect.setConfigProvider(ConfigProvider.fromMap(new Map([['HOST', 'localhost']]))),
        Layer.context(),
      ),
    },
    plugins: [EffectPlugin, RelayPlugin],
    relayOptions: {},
  });

  builder.queryType({});

  builder.queryField('roll', t =>
    t.effect({
      resolve() {
        return pipe(
          Effect.config(Config.withDefault(Config.string('HOST'), '<IP_ADDRESS>')),
          Effect.flatMap(host => Effect.succeed(host)),
          // What should I do for ConfigError?
          Effect.catchAll(() => Effect.succeed('none')),
        );
      },
      type: 'String',
    }));

  const schema = builder.toSchema();
  const document = parse(`{ roll }`);
  const result = await execute({ document, schema });

  expect(result.data).toEqual({
    roll: 'localhost',
  });

What does it type checks but does not seem to work at all meaning? I just feeling I misunderstood.

@iamchanii I have a bit more complex setup than your example. I tried your example and it sort of works. In my case, I am providing a layer to the effect field in its configuration. That layer depends on another layer, which depends on an another layer until the very last layer actually depends on a config. For some reason, I'm getting an error like this:
(Missing data at /secrets.MY_SECRET: "Expected /secrets_MY_SECRET to exist in the process context")

Even if I'm providing a config provider from a map, i.e., I'm expecting to see something more like this:
(Missing data at /secrets.MY_SECRET: "Expected /secrets.MY_SECRET to exist in the provided map")

This is confusing to me but irrelevant for the big picture. I'll try to water my use case down to a minimal reproducible code. However, my bigger concern is the following.
I would like this error to happen at deploy time instead of at runtime, i.e. when resolving the GraphQL requests. I want my application to fail the deployment if some configuration is missing. I'm not sure how this could be achieved, or if at all in this plugin but it's a crucial requirement I have nonetheless. The problem I see is that pothos-plugin-effect creates an effect runtime on every request. Somehow, I'd need that runtime to be built ahead of time and the resolution of all graphql requests would need to live within that single runtime. That would solve my issue, I believe. I just don't know how feasible that is.

I would like this error to happen at deploy time instead of at runtime, i.e. when resolving the GraphQL requests. I want my application to fail the deployment if some configuration is missing

In fact, there is very basic type inferrence for effect layers, contexts and services. getting layers as typed tuple and typing it is little bit difficult (at least for me) but I think this library have to do it.

The problem I see is that pothos-plugin-effect creates an effect runtime on every request. Somehow, I'd need that runtime to be built ahead of time and the resolution of all graphql requests would need to live within that single runtime.

Interest. How about this api?
effectOptions: {
  customRuntime: () => {
    return Runtime.make(...);
  };
}

Hey @iamchanii 👋 sorry for coming back to you so late. I was thinking about this plugin a lot these past few weeks. The whole layer thing didn't work for me conceptually. I believe it is an API design mistake to surface the construction of layers through the field definitions. In Effect, it is important that these layers get constructed when the application starts as opposed to at runtime. As an application author, I do not want to handle ConfigErrors at runtime. I want the deployment of my application to fail if the configuration is wrong or missing. Otherwise, such failures could lead to serious production incidents which could have been easily avoidable.
To figure out what the core value of this plugin is, I decided to remove it from my codebase and try to use some minimal implementation. This is what I ended up with (it's just pseudo-code, don't mind if it doesn't run):
When I initialize my app, I create an Effect runtime which I pass to the graphql resolvers via a modified context object.
// index.ts
const main = async () => {
  const runtime = await makeRuntime(); // imagine this returns an Effect runtime

  const yoga = createYoga({
    schema,
    (context) => ({ ...context, runtime }),
  });

  const server = createServer(yoga);

  server.listen(4000, () => {
    console.info('Server is running on http://localhost:4000/graphql');
  });
}

void main().catch(console.error);

I also created what I think the core behavior of this library is, a graphql effect resolver:
export const resolveEffect = <R>(
  runtime: Runtime.Runtime<R>
) => async <E, A>(
  effect: Effect.Effect<R, E, A>
) => {
  const exit = await Runtime.runPromiseExit(runtime)(effect);

  if (Exit.isFailure(exit)) {
    const cause = Cause.unannotate(exit.cause);

    if (Cause.isFailType(cause) && cause.error instanceof Error) {
      throw cause.error;
    }

    throw new Error(Cause.pretty(cause));
  }

  return exit.value;
}

And then, all I need to do is use it in my resolver like so:
resolver: (_parent, _args, context) => pipe(
  // imagine there's an Effect here,
  resolveEffect(context.runtime)
)

Having reflected on this issue, I've come to the conclusion that this plugin should do two things:

remove all mentions of layers to prevent its users from shooting themselves in the foot,
allow the user to pass in a reference to a runtime, not a runtime construction function, to make sure that this runtime is built at application startup time, not during the handling of a request. It should be a single runtime reference passed to all resolvers. Then the plugin could do the effect resolution for me as it does today but using the runtime if one was provided.

What do you think @iamchanii?

@vecerek I'm so appreciate for your feedback! first of all, this library started from just small idea so there may be design mistakes. but I think we can still take it right way. and then, we can release first major release.
I'm not sure if you know, recently I worked about refactoring and writing some documentations. and I got some thoughts as similar as you thought. Is really need to define services, contexts and layers in object field? and I am not experienced about Effect, but in general, I think it's enoguth to have a "main" Layer to context management. (And, hmm. I still think field-level context management maybe usefull some use cases nevertheless)

remove all mentions of layers to prevent its users from shooting themselves in the foot,

👍
Is this meaning for field-level's mentions of layers? just applied for layers? how aobut contexts and services?

allow the user to pass in a reference to a runtime, not a runtime construction function, to make sure that this runtime is built at application startup time, not during the handling of a request. It should be a single runtime reference passed to all resolvers. Then the plugin could do the effect resolution for me as it does today but using the runtime if one was provided.

👍
like so:
// $ExpectType Runtime.Runtime<...>
const runtime = await makeEffectRuntime();

const builder = new SchemaBuilder<{
  EffectRuntime: typeof runtime
  // or
  // EffectRuntime: Runtime.Runtime<...>
}>({
  plugins: [EffectPlugin],
  effectOptions: { runtime },
});

I think this is more simple than globalLayer and globalContext and this could be replaced these features. but I concerned that all users of this plugin will require knowledges about Runtime and make it. how do you think about it?

There seems to be some confusion around context/layer/services, they are different things:

a Layer is Constructor of Context, a Layer should only be built at boot time
a Context is an Object that carries a set of services
a Service is a value contained in a Context object

Generally speaking "field-level" provision of layer / context is useless, the sole reason to use context is to let the context bubble up through your app and provide it at the top, putting in the same place access of context and provision of context defeats the purpose of using context in the first place, a user could simply do whateverEffect.pipe(Effect.provide(context)) by themselves, no new api needed.
You really have only two choices:

you control initialisation and expose it as an effect itself so your final composed schema exposes some kind of "bindApp: Effect<ContextFromSchema, never, void>" and you internally run an Effect.runtime() during init to get a runtime to use when resolving queries (that carries context)
you don't control initialisation and rely on a user giving you a runtime that you use to execute

In a general sense to use Effect you need to know about Runtime so I wouldn't worry about a user that wants to use Effect but doesn't know Runtime

I fully agree with @mikearnaldi. Because of the design of pothos, controlling the initialization of the app is not achievable. Our only option is:

you don't control initialisation and rely on a user giving you a runtime that you use to execute

And about

but I concerned that all users of this plugin will require knowledges about Runtime and make it. how do you think about it?

I don't think Runtime is a hard-to-grasp concept. Also, if users already have a Layer (could be one that was created by merging many layers), creating a Runtime from that layer is as simple as applying Layer.toRuntime. Providing some code examples that show how runtimes can be created should be enough to address your concerns.

@mikearnaldi @vecerek thank you for your comments! I'm still learning about Effect, so I don't have it all figured out, but I'll share some thoughts on the plugin's implementation soon.

I just want sync my thoughts, and want to get a review from you.

Remove effect options from t.effect. and then, I would be able to remove many lines of type inference related code.
t.effect({
  type: 'String',
- effect: {
-   layers: [/* ... */],
-   contexts: [/* ... */],
-   services: [/* ... */],
-   failErrorConstructor: Error
- }
})

(fyi; failErrorConstructor was implemented from #2 and I think it is very rare use case. I'm considering offering it only as a plugin-level options. not field. cc. @XiNiHa )

When constructing SchemaBuilder, it will be accept runtime that the user wants to use globally. so, plugin doesn't control the runtime initialization and checking it is not the plugin's concern.
const builder = new SchemaBuilder({
  plugins: [EffectPlugin],
  effectOptions: {
+   runtime: appRuntime
+   // runtime: async (context) => appRuntime
  }
});

t.effect field resolver will do the following things:
declare const effectFieldResult: Effect<...>;

return Runtime.runPromise(effectOptions.runtime || Runtime.defaultRuntime)(effectFieldResult);

@iamchanii I think that looks good as a high-level plan
Regarding the details of:
declare const effectFieldResult: Effect<...>;

return Runtime.runPromise(effectOptions.runtime || Runtime.defaultRuntime)(effectFieldResult);

In the end, I still had to unwrap the error so that the pothos error plugin keeps working like so:
declare const effectFieldResult: Effect<...>;

return pipe(
  effectFieldResult,
  Runtime.runPromiseExit(effectOptions.runtime)
).then((exit) => {
  if (Exit.isFailure(exit)) {
    const cause = Cause.originalError(exit.cause);

    if (Cause.isFailType(cause) && cause.error instanceof Error) {
      throw cause.error;
    }

    throw new Error(Cause.pretty(cause));
  }

  return exit.value;
})

(fyi; failErrorConstructor was implemented from https://github.com/iamchanii/pothos-plugin-effect/pull/2 and I think it is very rare use case. I'm considering offering it only as a plugin-level options. not field. cc. @XiNiHa )

Although I agree that the customization ability is very rare to use, throwing the error cause as-is completely breaks the plugin behavior. Therefore I believe the plugin should expose a custom error type (like EffectFailureError?) to throw with a non-error cause.

(fyi; failErrorConstructor was implemented from #2 and I think it is very rare use case. I'm considering offering it only as a plugin-level options. not field. cc. @XiNiHa )

Although I agree that the customization ability is very rare to use, throwing the error cause as-is completely breaks the plugin behavior. Therefore I believe the plugin should expose a custom error type (like EffectFailureError?) to throw with a non-error cause.

Let's make one assumption: If the plugin user had follow pothos's error plugin introduction then Error interface and BaseError object type should have been defined.
How about just throwing Error which set cause field? if plugin user wants to check original error, could be add new field in BaseError object.
if (Cause.isFailType(cause) && cause.error instanceof Error) {
  throw cause.error;
}

throw new Error(Cause.pretty(cause), { cause });

We can decide to expose a custom error type but, it should be defined as object type somewhere by user or the plugin itself.

Update 2024: I recently worked on 1.0 of pothos-plugin-effect, and the API has changed in such a way that effectRuntime is now configured in SchemaBuilder. I expect that the initial cause of this issue will be resolved.
Thank you for the many discussions about the API in this issue. I wouldn't have been able to move forward without this discussion. If you get a chance, please take a look at my new work as well.


Unit tests and benchmarks for JsonMediaType

Unit tests and benchmarks for JsonMediaType
Description
Unit tests and benchmarks for JsonMediaType.
Motivation and Context
The current implementation is rather slow, this opens up for comparing alternative implementations.
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[x] I have added tests to cover my changes.

:+1:


Function To Center Design At Origin

Hi ! Is it possible to create a function, which centers the current design at (0,0) ?
I tried this:
def center_object(cell):
    p0, p1 = cell.get_bounding_box()
    centered = gdspy.Polygon(cell.get_polygons()).translate((p1[0]-p0[0])/2, (p1[1]-p0[1])/2)

    return centered

Here an example to try:
import gdspy
import numpy as np
import scipy.spatial

# To reset the libary every time you run the code
gdspy.current_library = gdspy.GdsLibrary()

def center_object(cell):
    p0, p1 = cell.get_bounding_box()
    centered = gdspy.Polygon(cell.get_polygons()).translate((p1[0]-p0[0])/2, (p1[1]-p0[1])/2)

    return centered

def main():
    top_cell = gdspy.Cell('top_cell')

    #Sample GDS
    unit_cell = gdspy.Cell('unit_cell')
    w = 20
    N = 2
    size = 2*w*N
    inner_rec = gdspy.Rectangle((-w/2-size/2,-w/2-size/2),
                                (w/2-size/2,w/2-size/2))
    outer_rec = gdspy.Rectangle((-w-size/2,-w-size/2),
                                (w-size/2,w-size/2))
    unit_cell.add(gdspy.fast_boolean(outer_rec,inner_rec,'not',layer =0))
    rec_array = gdspy.CellArray(unit_cell, N, N, (2*w, 2*w))

    # Center GDS
    centered = center_object(rec_array)
    # Add to Top Cell
    top_cell.add(centered)

    # Visualize
    gdspy.LayoutViewer()

if __name__ == '__main__':
    main()

Hello @totyped!
Instead of translating the polygons, it is easier (and safer) to move the reference origin:
import gdspy

def center_reference(reference):
    bbox = reference.get_bounding_box()
    # Move the reference origin to put the center of its bounding box at (0, 0).
    reference.origin -= 0.5 * bbox.sum(axis=0)
    # The reference is modified in place.
    # The same object is returned just for convenience.
    return reference

top_cell = gdspy.Cell('top_cell')

#Sample GDS
unit_cell = gdspy.Cell('unit_cell')
w = 20
N = 2
size = 2 * w * N
inner_rec = gdspy.Rectangle((-w / 2 - size / 2, -w / 2 - size / 2),
                            (w / 2 - size / 2, w / 2 - size / 2))
outer_rec = gdspy.Rectangle((-w - size / 2, -w - size / 2),
                            (w - size / 2, w - size / 2))
unit_cell.add(gdspy.fast_boolean(outer_rec, inner_rec, 'not', layer=0))
rec_array = gdspy.CellArray(unit_cell, N, N, (2 * w, 2 * w))

# Center CellReference or CellArray
center_reference(rec_array)

# Add to Top Cell
top_cell.add(rec_array)

# Visualize
gdspy.LayoutViewer()

Wow. This works really nicely !
I didn't know that you could modify the reference in place. Very cool.


lp: page selection with -P processed after number-up

Version: 1.3.9
CUPS.org User: martin.pitt.canonical
From http://bugs.debian.org/501311:
------ snip -------
lp -P 41-51 -o number-up=2 file.pdf
I expected it to extract pp. 41-51 from file.pdf, then convert them to
2up and send it to the printer.
I can think of two fixes:

In the description of the -P switch in the 'lp' manpage, explain
that the page numbers are those after page rearrangements.

Change the cups filters to first select pages with -P, then
rearrange pages.

The first fix clarifies the situation but makes it hard to use -P
with number-up, so I like the second fix.
------ snip -------
I confirmed the issue with 1.3.9.

@michaelrsweet said "IPP specifies the page-ranges applies after imposition, so the current implementation is correct."...
Where does IPP specify that page-ranges applies after number-up processing? I'm looking at
https://tools.ietf.org/html/rfc2566#section-4.2.7
and don't see it...
Anyway, this is super counter-intuitive. If it is really what the spec says, then it would be nice to also have an option like input-page-ranges which would allow users to select pages from the actual document.

You want to look at RFC 2911 - RFC 2566 (IPP/1.0) is OBSOLETE and experimental.

Thank you. Now, I'm looking at this:
https://tools.ietf.org/html/rfc2911#section-4.2.7
I don't see what you were referring to (back in 2008!).
In fact, section 15.3 says this:
  2. The document data is in the form of a print-stream in a known
     media type. The "page-ranges" attribute is used to select, as
     specified in section 4.2.7, a sub-sequence of the pages in the
     print-stream that are to be processed and images.

  3. The input to this step is a sequence of print-stream pages.
     This step is controlled by the "number-up" attribute. If the
     value of "number-up" is N, then during the processing of the
     print-stream pages, each N print-stream pages are positioned,
     as specified in section 4.2.9, to create a single impression.

I think the words "the input to this step" mean "the input to step 3", since steps are being numbered. I guess step 3 could be described as taking pages and turning them into "impressions". Impressions are sent to the printer and put on pages, but a single impression might contain multiple document pages. The way I read this, applying the "page-ranges" option (in step 2) takes the user-supplied range arguments and uses them to select numbered document pages (from the formatting step, step 1) not impressions (from step 3)...

@michaelrsweet - I didn't mean to scare you away with my interpretation of RFC 2911. I was just trying to help us both figure out where you were coming from... Were we looking at different documents? Different parts of the same document? Thank you.


Change the member function name from ConfigureSRC() to ConfigureSrc().

The Adau1361Lower::ConfigureSRC() is against the Google style guide.

For the purposes of the naming rules below, a "word" is anything that you would write in English without internal spaces. This includes abbreviations, such as acronyms and initialisms. For names written in mixed case (also sometimes referred to as "camel case" or "Pascal case"), in which the first letter of each word is capitalized, prefer to capitalize abbreviations as single words, e.g., StartRpc() rather than StartRPC().

https://github.com/suikan4github/rpp_driver/blob/2dc5c99cf581ce89b49e50063cfd91c915526631/src/codec/adau1361lower.cpp#L514
It has to be renamed to ConfigureSrc()

It was merged to develop. Ready to release.


Deletes cluster when upgrade fails

There is an issue I opened in etcd https://github.com/coreos/etcd/issues/8268 that highlighted what I think is a bug in expected behavior for the operator. When upgrading a cluster in this case, the operator deletes the entire cluster instead of reverting the cluster to the old etcd image.
It would seem preferable to me that the cluster enters a degraded state or reverts fully to the previous version instead of straight deletion. The reason for deletion in the logs is that no backup configuration exists in the spec.

So you mean when upgrade halted in the middle, revert to previous version?
To extend this topic, currently you can backup before upgrade and if anything goes wrong just restore fro it.

We can possibly add a timeout in upgrade.

The intention is that when upgrading, if upgraded members do not actually become part of the cluster the upgrade should be stopped or rolled back.

The intention is that when upgrading, if upgraded members do not actually become part of the cluster the upgrade should be stopped or rolled back.

Right. However, it is unplanned right now. Can you open a feature request for the upgrade rollback feature

Done


IFRAME doesn't display

Seems that the IFRAME's behaviour has changed. Using an API V3 token key I can no longer display the iframe. I get this display  :

the called URL is :  https://www.mailjet.com/campaigns?t=
in the URL I get a 404 error.
$params = array
(
"method" => "POST",
"AllowedAccess" => "campaigns,contacts,reports,stats,preferences,pricing,account",
"APIKeyID" => $ApiKeyID,
"lang" => "fr",
"TokenType" => "iframe",
"IsActive" => "true"
);
$result = $this -> m_oMailJet -> apitoken( $params );
$url = "https://www.mailjet.com/campaigns?t=".$result->Data[0]->Token;

Hi @bamboolix ,
First of all, thank you for using Mailjet!
Coud you contact our support (here) regarding this issue? This doesn't seem php-specific so they will be better suited to answer you.
Thanks in advance!

very well thanks for the Answer.

Hey @bamboolix :airplane:
We just released a new PHP API wrapper.
It is now more PHP-ish (including namespace, PSR-0 compliance, and globally a better architecture).
All it requires is PHP 5.4
In addition to our API Guides, we would be happy to help you get started with it.
Guillaume, from Mailjet!


ci: remove PAT and utilize a github app for tokens

Nx Cloud Report
CI is running for commit 2c08b76c6b44266cfe8e9b13fddf365a58fcf090.
📂 Click to track the progress, see the status, the terminal output, and the build insights.

Sent with 💌 from NxCloud.


PHPunit 6 update

phpunit PHPUnit_Framework_TestCase migration to namespace
bootstrap removed phpunit 4 case
setExpectedException changed to expectException
drop php5.5
update phpunit version
composer.lock added
drop hhvm from build matrix

While you're at it, set the fast_finish flag to false, please. :)

i removed it already =) with https://github.com/zendframework/zend-mime/pull/28/commits/3a025b2ea98d24d6402e56a15e34434422800183
i missed it=)

Thanks, @kokspflanze!


[SPARK-21574][SQL] Point out user to set hive config before SparkSession is initialized

What changes were proposed in this pull request?
Since Spark 2.0.0, SET hive config commands do not pass the values to HiveClient, this PR point out user to set hive config before SparkSession is initialized when they try to set hive config.
How was this patch tested?
manual tests

Test build #80042 has started for PR 18769 at commit 4bdc9ad.

Test build #80042 has finished for PR 18769 at commit 4bdc9ad.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80042/
Test FAILed.

Test build #80050 has started for PR 18769 at commit cee6838.

Test build #80050 has finished for PR 18769 at commit cee6838.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80050/
Test FAILed.

retest this please

Test build #80051 has started for PR 18769 at commit cee6838.

Test build #80051 has finished for PR 18769 at commit cee6838.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80051/
Test FAILed.

Merged build finished. Test FAILed.

Please describe the issue to fix in the PR description, rather than just the codes to reproduce.

@viirya Spark does not support that. see:https://github.com/apache/spark/pull/17223#issuecomment-286608743
@dongjoon-hyun How about throw exception when users try to change them as @cloud-fan said?
or document it?
I have test —conf spark.hadoop.hive.exec.max.dynamic.partitions=1001 both works for spark-shell and spark-sql, --hiveconf hive.exec.max.dynamic.partitions=1001 works for spark-sql.

Yep. The workaround has been the solution for this.
And, +1 for the exception according to @cloud-fan 's suggestion, too.

Test build #80114 has started for PR 18769 at commit 506050a.

SetCommand.scala throws exception seems roughly. InsertIntoHiveTable throws exception seems too late. so I logWarning at SetCommand.scala

Test build #80114 has finished for PR 18769 at commit 506050a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80114/
Test PASSed.

From the screenshot, it can't tell if the setting works or not. Please explicitly log that the setting doesn't work.

Test build #80155 has started for PR 18769 at commit 2eb39cb.

Test build #80155 has finished for PR 18769 at commit 2eb39cb.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80155/
Test PASSed.

@wangyum Could you help us submit a PR to fix the doc generation in https://spark.apache.org/docs/latest/configuration.html?
See many annoying syntax issues.

OK, I will try.

Test build #80297 has started for PR 18769 at commit 8787cbc.

You can refer the description in https://github.com/apache/spark/tree/master/docs to build the documentation. Thanks!

LGTM pending a few minor comments.

Test build #80298 has started for PR 18769 at commit 03f5753.

Test build #80298 has finished for PR 18769 at commit 03f5753.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Test build #80297 has finished for PR 18769 at commit 8787cbc.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80298/
Test FAILed.

Merged build finished. Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80297/
Test FAILed.

retest this please

Test build #80299 has started for PR 18769 at commit 03f5753.

Test build #80299 has finished for PR 18769 at commit 03f5753.

This patch fails SparkR unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80299/
Test FAILed.

retest this please

Test build #80302 has started for PR 18769 at commit 03f5753.

Test build #80302 has finished for PR 18769 at commit 03f5753.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/80302/
Test PASSed.

Thanks! Merging to master.

@gatorsmile Docs syntax issues was fixed by https://github.com/apache/spark/pull/18793.


Wrap installer in a function

This will assure that the install script will not
begin executing until after it has been downloaded should
it be utilized in a 'curl | bash' workflow.
Signed-off-by: Eric Windisch<EMAIL_ADDRESS>

LGTM
Ping @tianon

cool LGTM as long as those are tabs not spaces hehehe

Haha I'm with @jfrazelle on that endless debate :)

Those are all spaces indeed. :smile:

how can you tell?

mouse swipe it.. it will select by tab or space.. And, bash script, definitely want spaces there!! :)

oooooooooooo i see

should I cherry-pick this, or no? idk idk

why no we can test it on test.docker.com in the next Rc :)

Definitely not in its current form. :wink:

@jfrazelle yeah cherry-pick it! and yeah once the spaces are back to tabs :)
@tianon apart from the spaces/tabs debate, is there any other issue you see?

There... tabs

@ewindisch @tiborvass @tianon I think the alternative PR https://github.com/docker/docker/pull/12242 has now more "matured".
Should this PR be closed and removed from the 1.6 milestone (and https://github.com/docker/docker/pull/12242 added to the milestone)?
(and, please, make up and shake hands on the tabs vs spaces disputed :smile:)

Oh, sorry and @jfrazelle - your in charge for the release :)

I don't think either is more or less matured really, but don't care -- they're essentially identical.

@ewindisch I'll leave it up to @jfrazelle, just would like one closed, because this is confusing, and a waste if both need review :smile:

Sorry @ewindisch; I was looking at this one and saw a few more things I wanted to change that were easier to PR instead of describe. :angel:
Will rebase mine on top of yours when I get back to a computer so that the flow of credit is correct. :+1:

Closing in favor of #12242 that now has credit for @ewindisch :)


Added support for rippleColor in android

Part of this: https://github.com/okwasniewski/react-native-bottom-tabs/issues/11
Summary: This PR updates the ripple color of tab bar click for Android implementations.
Testing:
Verified Ripple color changes on Android.
Added FourTabsRippleColor for visual changes.
Adding Video for reference
https://github.com/user-attachments/assets/7cb8e2eb-b56c-4546-824a-315b11bd80ca

Thanks for working on this!
Can you check how the ripple color looks like when using Material Design 3?
I've added guide on how to change the styles here: https://github.com/okwasniewski/react-native-bottom-tabs/blob/docs/content-update/docs/docs/docs/guides/android-native-styling.md
(It doesn't work for expo yet, but it should work in Example App)

I have tested it using material design 3
this is how it looks
https://github.com/user-attachments/assets/c3d1f23b-673c-4245-85af-0741780ce93d


io_uring_register_buffers and io_uring_register_buf_ring

Does registering the buffers used in the buf_ring give performance improvements with multishot operations?

If we compare with the legacy provided buffers, i.e. IORING_OP_PROVIDE_BUFFERS, then yes, buf_ring is more performant, not needing to queue extra requests is a huge win.
But note that to use multishot receives in the first place you need one of those -- multishots potentially run indefinitely and you need a way to feed it buffers.

I didn't explained well, If i use io_uring_register_buffer on a buffer added to a buf_ring does It optimize the buffer operations?

I didn't explained well, If i use io_uring_register_buffer on a buffer added to a buf_ring does It optimize the buffer operations?

No, it does not. The kernel doesn't keep track of that association.
If one day there would be providing  registered buffers of some sort it should be explicitly reflected in the user API.  And nobody cared to implement it because it wouldn't help anyway for the same reason why registered buffers are useless for recv / recvmsg -- there is a memcpy inside the net stack.
A side note, there is a zerocopy recv in development, it requires registered buffers, and long merged send zc also benefits from registered buffers, but

Ok thanks!


about maintaining session continuity: how to get context by python clinet

If using an HTTP client, the context can be obtained from the last returned JSON of the previous generate request, and this context can be passed as a context parameter in the new generate, thus maintaining session continuity. However, when using a Python client, it is not possible to obtain the context in the JSON returned by the last request, making it impossible to implement this function. Does the Python client not provide this feature?

Hey! The pattern is a bit different when using the chat interface. Refer to this example here to see how you can attach assistant responses: https://github.com/ollama/ollama-python/blob/main/examples/chat-with-history.py


Change default label from 'wontfix' to 'stale'

Fixes #109 and #74.
Support for this change has been expressed by @patcon, @christianbundy, @sindresorhus, @paulmelnikow, @dwijnand, @darthmaim and @max-sixty in the issues linked above, and likely others who may have subscribed to them but haven't added comments or reactions to the discussion threads.

View rendered README.md

Thanks @waldyrious!
Alternative to the concern that stale isn't a default label: if the label doesn't exist, drop the message, and append a note to administrator that they'll need to create the stale label or set staleLabel to an existing label in order for functionality to work properly.
Not advocating that this necessarily needs to be within scope of this PR though

Alternative to the concern that stale isn't a default label: if the label doesn't exist, drop the message, and append a note to administrator that they'll need to create the stale label or set staleLabel to an existing label in order for functionality to work properly.

Is this true? I've never had to create the stale label, it gets created automatically when the label is added to an issue.

Oooo I didn't realize. Maybe I'm remembering old behaviour that was referenced in past issues that spoke against changing the default. But seems its been auto-created for at least 3 years
https://github.com/probot/stale/blame/71669525fd170d53194a4751fe967dc9c26a8a92/lib/stale.js#L204-L211

:/

🤦

this is fun

As a GitHub App with many existing installations, Stale's major risk in changing this default is that it will negatively impact existing repository installations that chose not to explicitly set the default label because they were happy with wontfix.
By changing the default here, those repository installations will be forced to either explicitly set that label now in their configuration now (:warning: without any warning, my you :warning:) or else future runs of Stale will creating a separate stale label and the behavior will become fractured as it won't honor wontfix labels that were applied to stale issues. 😕
Unfortunately, GitHub Apps don't have the luxury of opt-in versioning like, say, NPM package consumption or GitHub Action consumption does. If they did, I would suggest we just make this a breaking change (major version increment) per SemVer and ship it. However, without that capability, my personal inclination is to decline this change. 🤷🏻♂
@tcbyrd @JasonEtco @hiimbex Thoughts?

Thanks for engaging, @JamesMGreene :)
For a default that needed to change and folks agreed it was important, how would that happen? Is this what versioned config is for? Is there any precedent for allowing GitHub Apps to change defaults without pain, even when they're big.
Will admit (without judgement) that it feels a bit frustrating to hear "we're too big to do this now" after 3 years. ❤
As I've said elsewhere, being able to change defaults feels like a challenge worth engaging with, at least when we live imperfectly in a world where we'll almost surely make mistakes in deciding those defaults (not saying this is the case here).
Thanks!

@JamesMGreene if a significant portion of this app's users did prefer the current usage, I think it would be reasonable to expect at least some of that sentiment to have been manifested after almost 3 years since #74 was created — either there, in #109, or in this PR. Instead, what can be seen is overwhelming support for this change, both in the comments and in the reactions.
If anything, one could argue that the current behavior is more disruptive, since the bot removes the wontfix label when an issue has activity, which is problematic for the cases where the label was actually applied for its intended purpose, i.e. as a final state for an issue. That has been reported as #294.

I think @JamesMGreene said it right here. Part of the issue is that GitHub Apps don't have versioning, part of it is that introducing this kind of change is going to mess with a bunch of peoples' existing installations and require help from us.

Will admit (without judgement) that it feels a bit frustrating to hear "we're too big to do this now" after 3 years.

I get why this is frustrating. Probot, especially Stale, has seen folks come and go, and its now in a state where we'd rather not mess with it. I realize that doesn't sound great, but its the reality. It's in use by enough big orgs/repos that changing the behavior is going to cause a bunch of bugs and support requests that we simply can't deal with.
The original authors have left, there are better alternatives now (see actions/stale, and accepting significant changes means a big risk from the people (myself and the other folks mentioned above) that we'll have to put out fires for the users who have Stale installed.

Instead, what can be seen is overwhelming support for this change, both in the comments and in the reactions.

I don't mean to nitpick, but this isn't overwhelming support - the large majority of Stale users are fine with the default, or at least haven't told us otherwise. That's where configuration is important, and changing the default label shouldn't be an issue.
I'd also recommend sharing these concerns in actions/stale, a GitHub Actions version of Stale. They've chosen to set the default label as Stale.

#294 can be dealt with in a different and non-breaking fashion. Left a suggestion in that issue at https://github.com/probot/stale/issues/294#issuecomment-675743059

:roll_eyes:

Apologies to all of you who have expressed interest in this change but, alas, we are going to close it as wontfix (and yes, I absolutely see the irony) based on the discussion included here (namely, the unavoidable negative impact to existing installations).
I fully empathize with you all on this one. 💔  I agree that stale is a more practical default value to use but based on the historical configuration and large installation count, I also concur that it is best not to change that now. 😕
Also, my apologies that we did not close this sooner. 🙇🏻♂


rclone triggers error 500 on OCIS - might be an issue with permissions

Describe the bug
I run OCIS 1.14.0 on the server side and rclone 1.57.0 on the client.
OCIS runs with this parameter:
OCIS_BASE_DATA_PATH=/var/lib/ocis ACCOUNTS_DEMO_USERS_AND_GROUPS=false PROXY_HTTP_ADDR=<IP_ADDRESS>:443 OCIS_URL=https://storage.0x17.ch PROXY_TRANSPORT_TLS_KEY=./certs/privkey.pem PROXY_TRANSPORT_TLS_CERT=./certs/cert.pem ./ocis server

I use the following parameter for rclone:
./rclone -v --no-check-certificate --exclude-from exclude.txt -L sync /home/nicolas storage:

rclone will hang at some point and I see the following error in OCIS:
{"level":"info","service":"proxy","proto":"HTTP/1.1","request":"tresor/BeDSV2NlVa-001220","remote-addr":"<IP_ADDRESS>:33488","method":"MKCOL","status":500,"path":"/remote.php/webdav/.cert/nm-openvpn/","duration":27.317731,"bytes":0,"time":"2021-11-13T22:15:17Z","message":"access-log"}

I got rid of some other errors like this by removing the said file on the source and re-run rclone.
The pattern I think is relevant is, that the files that generate the 500 error on OCIS have permissions like this (on the source):
[nicolas][lunkwill][~/.cert/nm-openvpn] $ ls -l
total 16
-rw------- 1 nicolas nicolas 2780 Aug 18 11:10 nicolasc-ca.pem
-rw------- 1 nicolas nicolas 2533 Aug 18 11:10 nicolasc-cert.pem
-rw------- 1 nicolas nicolas 3243 Aug 18 11:10 nicolasc-key.pem
-rw------- 1 nicolas nicolas  636 Aug 18 11:10 nicolasc-tls-auth.pem

-> There are no permission flags for the group and others set.
Is it possible, that this (missing group/others permissions) cause a 500 error in OCIS when using rclone?

I think I found a way to reproduce my issue; I'll create a better bug report and therefore close this one.


Make LICENSE.txt consistent with documentation

The license (3-clause BSD) contained in the documentation under doc/getting_started/license.rst was not the same as the content of LICENSE.txt
This PR adds the 3-clause BSD license to LICENSE.txt, and includes that file directly into the license.rst with an include statement.
Also, the CREDITS section is updated to include the new collaboration, and the citation is updated to use the DOI created by Zenodo.

I tried to update the classifier for the license to a more specific one, but I think it will fail to upload to pypi if it doesn't match one of the ones here:
https://pypi.org/classifiers/ so I reverted to "License :: OSI Approved :: BSD License"


Contracted Network has routing issues.

Hi
I am having some issues with routing when using a contracted network.  When using a contracted network I am seeing issues where the route is very strange.  See screenshots below with comparison of route section with contracted and non-contracted network.
The OSM data I am using is: "new-zealand-latest.osm.pbf"
Code:
using Itinero;
using Itinero.IO.Osm;
using Itinero.Osm.Vehicles;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace Itinero
{
    class Program
    {
        static void Main(string[] args)
        {
            var routerDb = new RouterDb();
            /*  using (var stream = new FileInfo(@"c:\tmp\new-zealand-latest.osm.pbf").OpenRead())
              {
                  routerDb.LoadOsmData(stream, Vehicle.Car); // create the network for cars only.
              }

              //routerDb.AddContracted(Vehicle.Car.Fastest());

              using (var stream = new FileInfo(@"c:\tmp\new-zealand-nc.routerdb").OpenWrite())
              {
                  routerDb.Serialize(stream);
              }
             */

            using (var stream = new FileInfo(@"c:\tmp\new-zealand-nc.routerdb").OpenRead())
            {
                routerDb = RouterDb.Deserialize(stream);
            }

            // create a router.
            var router = new Router(routerDb);

            // get a profile.
            var profile = Vehicle.Car.Fastest(); // the default OSM car profile.

            // create a routerpoint from a location.
            // snaps the given location to the nearest routable edge.
            var start = router.Resolve(profile, -43.5321f, 172.6362f);
            var end = router.Resolve(profile, -45.031162f, 168.662643f);

            // calculate a route.
            var route = router.Calculate(profile, start, end);

            using (var sw = new StreamWriter(@"c:\tmp\points-nc.txt"))
            {
                route.WriteGeoJson(sw);
            }
        }
    }
}

Contracted:

Non contracted:

Ouwch, that looks bad, I'll have a look at this.

I just tried this out, I can't seem to reproduce this, I get identical routes in both cases:
http://geojson.io/#id=gist:xivk/a952bb08244b9dc9254205880b1afbdf&map=8/-44.301/170.650
http://geojson.io/#id=gist:xivk/3a98c9f5e2f97a9ab18b882cf6179176&map=8/-44.301/170.650
Did you use the default profile? Perhaps it only works on the New Zealand file from yesterday? What version did you use to test? The latest prerelease of the stable?

I have just tried with the latest OSM file and the issue does not reproduce.  Perhaps that is due to a difference in the ordering of the data in the file.  The issue only happens when contraction is enabled so it appears to be something related to the way the nodes are being contracted.
This data file reproduces the issue:
https://www.dropbox.com/s/x25mnsahogkop2g/new-zealand-latest.osm.pbf?dl=0
I am using version 1.4.0-pre44.

Ok, thanks, I'll have a look. It's definitely possible that this only happens with a specific file. It's going to take me some to figure this out.

Narrowed this down to an issue with the combination of turn-restrictions and edge-based contraction. The route is fine when using an uncontracted network and a contracted network without turn-restrictions. The route has the issue above when using a contracted network combined with turn-restrictions.

Is there a fix for issues like this? I get issues like this on my routings all the time, especially with the Classifications profile, sometimes it does a turn from highway trunk to residential and then back to the highway trunk for no reason.. :/


[WIP] feat: 0.8.22 support

[x] macOS bins updated
[x] Latest set to 0.8.22
[ ] https://github.com/nikitastupin/solc updated to 0.8.22

I'm attempting to trigger the build and publish action for the v0.8.22 release , but it appears that only the maintainer has the ability to initiate it


proposal: x/sys/unix: add System V shared memory functions

What did you expect to see?
It would be mildly useful to have shmget/shmat/shmdt/shmctl available in pure go. System V shared memory isn't very common in modern applications, but there are still uses for it that can't be accomplished otherwise. Usages I'm aware of:

X11 MIT-SHM extension uses shm to exchange image data
The American Fuzzy Lop fuzzer uses shm as a mechanism to communicate with the program being fuzzed.

I think the primary question I'd like to have answered is whether inclusion is appropriate for x/sys/unix and how likely it would be for it to be accepted, given that its usage is fairly esoteric.
Prior art
There are several cgo implementations I had found previously, but my google-fu is failing me at the moment. I had proposed an initial API at https://github.com/golang/sys/pull/108 , but hadn't noticed the other pure go prior art in my initial search. There are several mostly pure go implementations:

https://github.com/gen2brain/shm provides a consistent higher level API for all of its supported OS, only requires cgo on solaris (see https://github.com/golang/go/issues/24357)
https://github.com/nvi-inc/svipc is linux only and provides raw syscalls only

cc @gen2brain

/cc @jayconrod in case fuzzing has run into this as well

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

I think this would be good to have. I looked for this before deciding how to build our fuzzing shared memory implementation and was sad not to find it. Ultimately, I went with a memory mapped file to minimize platform differences, but it would be nice not to have to write anything to disk.

Keeping in mind Ian's feedback on my initial PR:

Most additions to this repo are fairly mechanical expressions of underlying Unix concepts, and we just accept them.  This is more complex, and I think it ought to go through the proposal process.

If this were to be accepted + implemented, there are probably several ways this could be implemented. I vote to follow gen2brain/shm's approach instead of my original PR, as it's more of a mechanical expression of the underlying API. For reference, here's the original C functions copied from linux glibc:
/* Shared memory control operation.  */
extern int shmctl (int __shmid, int __cmd, struct shmid_ds *__buf) __THROW;

/* Get shared memory segment.  */
extern int shmget (key_t __key, size_t __size, int __shmflg) __THROW;

/* Attach shared memory segment.  */
extern void *shmat (int __shmid, const void *__shmaddr, int __shmflg)
     __THROW;

/* Detach shared memory segment.  */
extern int shmdt (const void *__shmaddr) __THROW;

shmget/shmdt
shmget and shmdt can be straightforward autogenerated wrappers around the syscall in the normal fashion.
shmat
shmat syscall does not provide the size of the returned memory region. gen2brain/shm's implementation of shmat calls the syscall to get a pointer to the memory region, and then calls another syscall to query the size of the region. It then uses the size to construct the byte slice header. Should an x/sys/unix implementation:

Use this approach (arguably safer, but at the expense of another syscall)
Require the user to pass in the size of the slice to be created and hope they get it right (this is low-level functionality after all?)
Provide a means for the user to do both?
Return a uintptr and let the caller decide what they want

I prefer 1 or 3. Converting to a byte slice is likely the common case, and follows how mmap works.
shmctl
shmctl can use the normal autogenerated syscall wrappers, but has slightly different control structures for each OS. Should an x/sys/unix implementation:

Just autogenerate the appropriate struct for each OS and let users deal with the differences using build tags (this is how gen2brain deals with it)
Try to unify the structs with a common interface for features present on all OS, and then have a Sys() function that returns the underlying OS-specific struct (as os.ProcessState does)

I lean towards 1 as I don't think the struct argument is used all that often.
helper functions in gen2brain/shm
There are two really useful helper functions that gen2brain added in addition to the standard sysv API

Rm: removes the shared memory segment (wrapper for shmctl(IPC_RMID))
Size: returns the size of the shared memory segment

Given Ian's initial comment, I imagine these two functions would not be adapted to x/sys/unix as they're straightforward to implement if you know how the shm API works.

It would be nice to have a more standard API that hides some of the per-OS variation, like syscall.Mmap and syscall.Munmap.
The gen2brain/shm API looks reasonable, although it would be good if we didn't need fields like PadCgo and GlibcReserved4.
The gen2brain/shm API is:
func At(shmId int, shmAddr uintptr, shmFlg int) (data []byte, err error)
func Ctl(shmId int, cmd int, buf *IdDs) (int, error)
func Dt(data []byte) error
func Get(key int, size int, shmFlg int) (shmId int, err error)
func Rm(shmId int) error
func Size(shmId int) (int64, error)
type IdDs
type Perm

For x/sys we would need more prefixes, like:
func SysvShmAttach(id int, addr uintptr, flag int) (data []byte, err error)
func SysvShmCtl(id, cmd int, desc *SysvShm) (int, error)
func SysvShmDetach(data []byte) error
func SysvShmGet(key, size, flag int) (id int, err error)
type SysvShm
type SysvIpcPerm

The _ds in shmid_ds apparently stands for data structure, so shortening shmid_ds to Shm seems OK.
The Rm and Size helpers seem to be wrappers around shmctl.
The Sysv prefixes are to avoid confusion with POSIX shm_open and shm_unlink if those are ever added
(Those would be PosixShmOpen and PosixShmUnlink.)
Thoughts?

The proposed API seems mostly fine to me, with two caveats.
The shmid_ds seems to me to be a shorthand for 'descriptor', not 'data structure' -- though I can't find a citation for it either way. In either case, SysvShm seems like a bad name because to me it implies that it is the shared memory itself. It just describes it. How about SysvShmDesc instead?
It seems like most other API in sys follow the unix names pretty closely, so this might make it slightly less discoverable. However, it might be Good Enough to just make sure that the unix names show up in the documentation. I don't have a strong objection to it.
Answering my final question then:

Should an x/sys/unix implementation:

Just autogenerate the appropriate struct for each OS and let users deal with the differences using build tags (this is how gen2brain deals with it)
Try to unify the structs with a common interface for features present on all OS, and then have a Sys() function that returns the underlying OS-specific struct (as os.ProcessState does)

Your API is option number 1. I think it will be the easiest to implement, so I agree.

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

Initial PR @ https://github.com/golang/sys/pull/110, feedback welcome. I chose SysvShmDesc as stated in my objection since nobody objected to my objection, and it's a clearer name than SysvShm.

It would be good if someone with the various OS/arch combinations that golang supports were able to run the build scripts to autogenerate the various z* files. I only have OSX and Linux setup locally.
I tried adding linux support by running GOOS=linux GOARCH=amd64 ./mkall.sh which runs the docker container... and the diffs are really quite large. Is there a step I'm missing?

Thanks, people can open specific issues for specific platforms (or just send a patch).


docs(notes): add hint to sucessfully commit a contribution

ref.: https://github.com/isaachinman/next-i18next/issues/234

I think I'd rather add a "How to contribute" section to the docs instead - will track that via #237. Thank you though!

@isaachinman np :) thank you for your collaboration!


How do I get the remaining days left for the Maximum password age to expire?

I need it for monitoring and informing the users their Maximum password age's remaining days.

@jgoc
The issues section of this repository is intended for documentation and content related issues.
Please open a support ticket via the link below so that that your question gets answered quickly.
Windows 10 Support
Thank you.


HTTP: Premature close in Gatling 3.3.1

Hi, I've just upgraded to 3.3.1 from 2.3.0. Everything worked okay with the exception of one scenario. Here is a minimal example:
package upgrade

import io.gatling.core.Predef._
import io.gatling.http.Predef.http

import scala.concurrent.duration._

class TokenInfoSimulation extends Simulation {

  private val baseProtocol = http
    .userAgentHeader("Gatling")
    .acceptEncodingHeader("gzip,deflate")

  private val tokenInfoScenario =
    scenario("Token Info")
      .exec(
        http("Token Info")
          .get("https://access.stage.api.bbc.com/v1/user/tokenInfo")
      )

  setUp(
    tokenInfoScenario
      .inject(
        constantUsersPerSec(5) during (10 seconds)
      )
      .protocols(baseProtocol)
  )

}

All of the requests consistently fail with Premature close:
2020-09-08 14:34:43                                          10s elapsed
---- Requests ------------------------------------------------------------------

Global                                                   (OK=0      KO=50    )
Token Info                                               (OK=0      KO=50    )
---- Errors --------------------------------------------------------------------
j.i.IOException: Premature close                                   50 (100.0%)

And here is the log:
simulation.log
Not sure if this is useful, I was previously on:
scalaVersion := "2.11.7"
libraryDependencies += "io.gatling" % "gatling-test-framework" % "2.3.0" % "test"

and now:
scalaVersion := "2.12.0"
libraryDependencies += "io.gatling" % "gatling-test-framework" % "3.3.1" % "test"

Could you please share a sample target app? Without one, there's no way to execute a test that hits "https://localhost/v1/user/tokenInfo".

Thanks for getting back to me! I've updated the example

It looks like this issue was present in 3.3.1 but was already fixed in master/upcoming 3.4.0.

Awesome, thanks!

Pretty swamped before the upcoming release train, so I don't have time to look for the duplicated issue :(

Duplicate of #3746


Debug Repl: output of same value no longer aggregated

I am not sure when we lost support for this but we used to show a little number next to the output if the output was identical to the previous output, much like Chrome console works:

I am no longer seeing this in the repl:

Please note that in your picture you are outputing an object which we never supported. Though what you are doing in chrome picture should work
for (var i = 0; i < 20; i++) {
    console.log("Hello world");
}

running the for loop from above with the "legacy" protocol results in this:

running it with "inspector" protocol in this:

reopening.

The issue is how the events are coming from the node extensions, the following sample shows that everything is fine on the vsocde side
setInterval(() => {
    console.log("Hello world");

}, 50)

However the legacy adapter seems to append different pieces together when they are happening shortly afterwards.
I do not plan to fix this on the vscode side as I would have to split the output which is coming together by newlines which would bring me some other pain.
@weinand if you want you can tackle it on the adapter side, closing this as I plan to take no action

I see.
Could you please explain what the blue number means for en end user?

I'm in favour of disabling this feature for now because it does not provide a user experience that is understandable. Compare the console in Chrome Dev Tools with both VS Code protocols to understand what I mean.
E.g. compare the output of this snippet in the "inspector" protocol and Chrome Dev Tools:
for (var i = 0; i < 20; i++) {
    console.log("Hello\nWorld");
}

I am fine disabling it. @bpasero objections?

No objections as I am not using this feature. Btw I verified this using extension debugging where it seems to work fine.

To verify: quickly try out if the repl properly shows strings sent from the adapter. And verify it does not concatanate them

Verified for EH debugging.


Is there any live demo for Orchard2 ?

There is almost nothing to show at the moment, you can barely Setup an instance. BTW you should join Gitter if you want to talk about it.

https://gitter.im/OrchardCMS/Orchard2

do you have any plan to provide a live demo for orchard core? especially now that almost reached beta3?


Restrictions in outgoing traffic

Action step-security/harden-runner helps prevent known security breaches in CI/CD environments.
By default, it restricts DNS traffic which causes failure during VM setup:
Setting up VM
exec ssh: mkdir -p /home/runner/work && ln -s /home/runner/work/  work
/usr/bin/bash /home/runner/work/_actions/vmactions/freebsd-vm/v1.0.4/run.sh execSSH
Config file: freebsd-14.0.conf
Pseudo-terminal will not be allocated because stdin is not a terminal.
exec shell: bash run.sh installRsyncInVM
/usr/bin/bash
Config file: freebsd-14.0.conf
Updating FreeBSD repository catalogue...
pkg: No SRV record found for the repo 'FreeBSD'
pkg: packagesite URL error for pkg+http://pkg.freebsd.org/FreeBSD:14:amd64/quarterly/packagesite.pkg -- pkg+:// implies SRV mirror type
pkg: packagesite URL error for pkg+http://pkg.freebsd.org/FreeBSD:14:amd64/quarterly/packagesite.txz -- pkg+:// implies SRV mirror type
Unable to update repository FreeBSD
Error updating repositories!
exec shell: bash run.sh showDebugInfo

I've found, that changing mirror_type and URL format in package repos config should help, but I didn't verify it.
Steps to reproduce
Minimal reproducible example:
name: Hardened Runner & FreeBSD
on: [push]

jobs:
  test:
    name: Test
    runs-on: ubuntu-latest
    timeout-minutes: 10

    steps:
      - name: Harden Runner
        uses: step-security/harden-runner@eb238b55efaa70779f274895e782ed17c84f2895 # v2.6.1
        with:
          egress-policy: audit

      - name: Run command inside VM
        uses<EMAIL_ADDRESS>        with:
          run: |
            echo 'It works!'

please try again with the latest release.

I can confirm that with version v1.0.5 (d139f0eaa5d47d9fcc44f7ab1748574475d89565) I can use step-security/harden-runner action.
Thanks for very fast response!


docsite: build documentation for "latest" and "release-0.x"

We have an initial version of the docsite: https://intel.github.io/pmem-csi/
It currently redirects to: https://intel.github.io/pmem-csi/latest/README.html
Once we branch for 0.7.0, we also want https://intel.github.io/pmem-csi/0.7/ and link to it from https://intel.github.io/pmem-csi/latest/README.html.
The linking part can be solved by updating the content of the .md file on "devel". Publishing will require some more work: right now, https://github.com/intel/pmem-csi/blob/6e79011642dc184e8e4f487cda667b505759d9ab/.github/workflows/publish.yml#L36 always replaces the content of "gh-pages".
We probably want to keep that approach because we can then do sweeping changes across the entire web site. What we need to add is iterating over all active branches and accumulating that output.

See https://github.com/intel/pmem-csi/pull/629.
Before finally closing this issue, let's fork 0.7 (currently the release-0.7 branch is identical to an older devel) and in the top-level README.md insert v0.7.x after "Introduction to PMEM-CSI" - that way one can see at a glance that this is versioned documentation.


not able to load model using torch

i am loading the model and trying to convert the model into onnx , but i was getting this error. could you please tell me what's issue here?
pytorch version = 1.9.0+cu111
python version = 3.8.16
checkpoint = torch.load("/home/neoastra/Downloads/raf-db-model_best.pth")   File "/home/neoastra/miniconda3/envs/yoga_val/lib/python3.8/site-packages/torch/serialization.py", line 607, in load     return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)   File "/home/neoastra/miniconda3/envs/yoga_val/lib/python3.8/site-packages/torch/serialization.py", line 882, in _load     result = unpickler.load()   File "/home/neoastra/miniconda3/envs/yoga_val/lib/python3.8/site-packages/torch/serialization.py", line 875, in find_class     return super().find_class(mod_name, name) AttributeError: Can't get attribute 'RecorderMeter1' on <module '__main__' from 'onnx_conversion.py'>

I get the same error. Can you elaborate on the solution?
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
[<ipython-input-3-14510c58e132>](https://localhost:8080/#) in <cell line: 1>()
----> 1 model = torch.load('/content/affectnet-7-model_best.pth')

2 frames
[/usr/local/lib/python3.10/dist-packages/torch/serialization.py](https://localhost:8080/#) in find_class(self, mod_name, name)
   1163                     pass
   1164             mod_name = load_module_mapping.get(mod_name, mod_name)
-> 1165             return super().find_class(mod_name, name)
   1166
   1167     # Load the data (which may in turn use `persistent_load` to load tensors)

AttributeError: Can't get attribute 'RecorderMeter1' on <module '__main__'>


marshalling does not apply defaultNamespaceRemap setting to nested elements

When marshalling GeneralAddress with nested xml element and having org.glassfish.jaxb.defaultNamespaceRemap property set to a custom value, ie http://test.com/ns, nested element is being put to the default namespace while the root one is put to the custom one. Expectation is that both elements appear in the same namespace.

    @XmlAccessorType(XmlAccessType.FIELD)
    @XmlType(name = "")
    @XmlRootElement(name = "generalAddress")
    public static class GeneralAddress implements Serializable {

        private final static long serialVersionUID = 1L;
        protected Address address;

        public Address getAddress() {
            return address;
        }

        public void setAddress(Address value) {
            this.address = value;
        }

    }

    @XmlAccessorType(XmlAccessType.FIELD)
    @XmlType(name = "")
    @XmlRootElement(name = "address")
    public static class Address implements Serializable {

        private final static long serialVersionUID = 1L;
        protected String list;

        public String getList() {
            return list;
        }

        public void setList(String value) {
            this.list = value;
        }

    }

so what is the correct behaviour? For now in version jaxb-runtime-4.0.3 nested elements don't inherit the root element namespace even if i don't use defaultNamespaceRemap .


Update pull request template

Remove /kinds that don't make sense for PR
Possible kinds are as quotes to prevent labeling with everything when someone doesn't read this.

small fix -> self approve ;-)
/approve

/lgtm


Thoughts on Bridgetek EVE (Embedded Video Engine)

Have you encountered displays using the Bridgetek EVE[234] controllers?
https://github.com/RudolphRiedel/FT800-FT813  (C driver, MIT License)
https://www.ftdichip.com/EVE.htm
Would these be difficult to incorporate into luma.lcd, for, say, a new Python programmer -- asking for a friend :)  There's a 4.3" IPS display from Newhaven Display that looks like it could be a nice upgrade from the TN panel I've been using:
https://www.newhavendisplay.com/nhd43800480ftcsxpctp-p-9663.html
The resolution is certainly high (800 x 480), but my use model mostly consists of static information (just a progress bar and elapsed timer changing most of the time).  High frame rates would not be a goal.

Short answer is no, I hadn’t seen these display controllers before. I had always resisted supporting higher resolution devices because of the slow frame rates that a python implementation would yield.
However, since the advent of the latest diff_to_previous() changes, this may not be such a huge issue, but it would still mean that we would need to write and test any new driver code.
An alternative is that if there is already a kernel framebuffer implementation for this device (i.e. it exports a /dev/fb0 device), then we can simply use the luma.core.device.linux_framebuffer() pseudo-display right now.

An alternative is that if there is already a kernel framebuffer implementation for this device (i.e. it exports a /dev/fb0 device), then we can simply use the luma.core.device.linux_framebuffer() pseudo-display right now.

Ah, I had not realized that functionality existed as part of luma.core.  For display through /dev/fb* I was thinking one had to switch to something like Pytorinox's framebuffer.py.  Good to learn something new!
I'll need to review the documentation for those EVE controllers further.  They seem quite a bit different than other controllers.  The marketing for the displays targets fairly low-powered hardware (like Arduino's), with EVE offering extensive onboard support for rendering text, drawing primitives, etc.  I think the display controller ends up being more powerful than the connected microcontroller!  I still need to see if they even offer the usual "make the pixel at this location this color" capability. :)

Version 1.2 of Bridgetek's FT81X Series Programmer's Guide can be found here:
https://brtchip.com/wp-content/uploads/Support/Documentation/Programming_Guides/ICs/EVE/FT81X_Series_Programmer_Guide.pdf
Perhaps I'm missing it, but I don't think the EVE controllers offer a pixel-based framebuffer for external use.  Most of the programming guide is devoted to the high-level commands one can issue (Section 4).  One can transfer bitmaps, with compression.  I don't know enough to recognize whether that forms a viable alternative drawing path for Pillow images or not!
It's fine with me if you want to close this out.  I'd just thought I would explore a bit.

Well, if it supports sending bitmaps, then we can make it work - after all that is all that the device.display(image) method does. The image parameter is a Pillow image type, but of course it can be serialized into a JPEG or PNG byte stream and sent to the device

https://brtchip.com/bt81x/
may be a nicer starting point.  There are links from that page to lots of demos, and the BT81x datasheet, as well the the Programmer's Guide.

Well, if it supports sending bitmaps, then we can make it work

RAM_G appears to be a 1 MiB space that is the device's general purpose graphics RAM (main memory).  SPI commands can write directly there.  In addition, there are several coprocessor commands that seem geared to assisting with image copying / decompression:

CMD_INFLATE.  This command is used to decompress the [subsequent] compressed data into RAM_G.  The data should have been compressed with the DEFLATE algorithm, e.g. with the ZLIB library.
CMD_INFLATE2.  This command is used to decompress data into RAM_G.  The data may be supplied in the command bufer, the media FIFO, or from [the onboard] flash memory.
CMD_LOADIMAGE.  This command used to load a JPEG or PNG image.  The image data should be in the following formats: regular baseline JPEG or PNG with bit-depth 8 only and no interlace.

The majority of the coprocessor CMD_* commands are for the built-in GUI widgets (sliders, dials, keys, gradients, etc).
Once in memory, there are several related Display List commands:

BITMAP_HANDLE.  Specify the current bitmap handle.
BITMAP_SOURCE.  Specify starting address in RAM_G for current bitmap.
BITMAP_SIZE and SIZE_H.  Specify screen drawing width, height for current bitmap.
BITMAP_LAYOUT and LAYOUT-H.  Specify bitmap pixel format for current bitmap.
BEGIN.  Seemingly needed to start the drawing.

So, maybe it would be possible.  The big thing I'm uncertain about is how one goes about updating an existing bitmap safely (i.e., without getting any bad artifacts onscreen).  Can one update an existing bitmap in-place?

I went ahead and ordered the Newhaven display I mentioned at the start of this thread.
It should hopefully be fun to play with.  More fun, anyway, than wrestling with DRM/KMS applications to share a GPU to drive an HDMI display!  :)

I went ahead and ordered the Newhaven display I mentioned at the start of this thread.

Scratch that.  Upon reading through the programming guide in more detail, I'm afraid it's not well-suite to my application.  There are indeed two ways of getting a bitmap onto the FT813 -- either transfer the data directly into RAM_G or use one of the co-processor commands (CMD_INFLATE, CMD_LOADIMAGE) to handle decompression and the jpg or PNG format on your behalf.  However, all of the RGB pixel formats have fairly limited color-depth!  The highest appears to be 16 bits.
There is a PALETTED8 format to handle an indexed-color bitmap.  The indexed colors are each 32-bits (full ARGB, 8 bits for each channel including alpha), but I'm pretty sure the bitmap is then limited to having at-most 256 distinct colors.
The RAM_G area is only 1 MiB in size.  For either 800x480 or 800x600 (the controller's maximum resolution), that is only sufficient space for 16 bits per pixel.
I think The EVE series of controllers really is intended more for applications in which one is mainly using the controller's built-in widgets or a set of small(ish) bitmaps to construct a GUI or information panel.  Given the color depth restrictions, the series doesn't seem well suited for true color bitmaps.
So, I guess my search for a SPI-connected IPS panel of decent resolution and size (>3.2 inches) has to continue.
Thanks to @rm-hull for entertaining my wild thought.

I did actually send off a request for an evaluation kit, but got a generic brush-off:

Hello Richard,
Thank you for your interest in Newhaven Display.
Due to recent changes to our policy, unfortunately we cannot provide free of charge samples.
Samples may be purchased directly from our website or through one of our distributors such as Digi-Key or Mouser Electronics.
Please let me know if you have any questions.
Best Regards,
S----- M------  |  Technical Support
Newhaven Display International, Inc.
2661 Galvin Court, Elgin, IL 60124 - USA
Call:<PHONE_NUMBER>  |  Fax:<PHONE_NUMBER>
www.newhavendisplay.com

I did actually send off a request for an evaluation kit, but got a generic brush-off

Well, it worth a shot! :)
Hunting around further for anything ellse "interesting" that's SPI-connected, there is this product. built around an RA8876 and intended for use as an Arduino shield:
http://www.techtoys.com.hk/BoardsKits/HDMIshield/HDMIshield.htm
The RA8876 looks to have full 24-bit color depth, depending upon how much SRAM one is willing to throw down on the board.  It has a general purpose framebuffer, well more than one actually, with HW assist for picture-in-picture and (I think) panning.  There are a few graphics primitives as well, but different than the EVE line.  The support for true color RGB888 pixels is appealing.
The price is a little high, though, particularly when combined with that of decent HDMI panel.
Adafruit has a somewhat similar breakout board / adapter based upon the RA8875, for directly driving a TFT panel.  That controller looks to be limited to at-most 16-bit color depth, though.
I started looking at such adapters after getting frustrated with a 4" IPS HDMI panel I was hoping to use with an RPi4.  Unfortunately, the modern GPU drivers for Pi -- well, Linux generally -- don't seem keen on providing distinct framebuffers for each head.  I think I'll just have to wait to see what develops.
In the meantime, I'm going to try a 3.5" IPS SPI-connected 480x320 panel with an ILI9486L controller.  That controller seems a fairly close relative to the ILI9341 (18-bit color depth, SPI transfers look to be the same), so I think I can get it working with luma.lcd with relatively few changes.
(If you haven't noticed, I really want to move to an IPS panel!)

@rm-hull  -- bump just for last update.  Since I posted to an already closed issue, I thought no notification might get sent.  I thought you'd appreciate the content, though.

@rm-hull  -- bump just for last update.  Since I posted to an already closed issue, I thought no notification might get sent.  I thought you'd appreciate the content, though.

Cheers @mattblovell ... good to know
a lot of my GitHub notifications (inc yours) seem to just go straight to the spam folder even though I’ve may times told gmail that they aren’t spam
-sigh-


A new file added to list important points in Data Structures

Important points should be explained in this file.

Pull request has already been merged