feat(javascript): Meta string encoding algorithm for JavaScript

What does this PR do?
Meta string encoding algorithm for JavaScript

#1542

Does this PR introduce any user-facing change?

[ ] Does this PR introduce any public API change?
[ ] Does this PR introduce any binary protocol compatibility change?

Benchmark

Hello @theweipeng ,  A metastring encoding and decoding algorithm has been implemented in javascript but I don't know hat else to do. Please have a look at my draft.

Hello @theweipeng ,  A metastring encoding and decoding algorithm has been implemented in javascript but I don't know hat else to do. Please have a look at my draft.

Great work! Since we are using TypeScript in this project, could you please rename the file to have a .ts extension and resolve the TSLint issues?

Do we need to merge those three classes into one file named metastring.ts?

I went through the https://www.conventionalcommits.org/en/v1.0.0/ but I still manage to fail the lint test.

Do we need to merge those three classes into one file named metastring.ts?

I am still studying the structure of the apache furry code base.  I don't really know much but I think it's best to use the same file structure as the ones in the java  code base , so that it will be easier for new developers to navigate through.

Hi, @chaokunyang , please can you  tell me how to pass this test https://github.com/apache/incubator-fury/actions/runs/9441081022/job/26001294763?pr=1675

CI failed with:

@theweipeng could you take a look at it

Hi  i did the npm run test  , but my logs are different @chaokunyang  , please can you tell me the command to typed in to get those logs ?

Hi i did the npm run test , but my logs are different @chaokunyang , please can you tell me the command to typed in to get those logs ?

If you use VSCode to edit the TypeScript code, you will see type errors underlined. Alternatively, you can navigate to the packages/fury  directory and run npm run build; typically, you can see compilation errors there.
I cloned your branch, and after executing npm install and npm run test, I was able to reproduce the errors on my computer.

Code style check faileld:

Seems we have not add code lint document for javascript, @theweipeng could you update the lint document?

I went through the https://www.conventionalcommits.org/en/v1.0.0/ but I still manage to fail the lint test.

Do we need to merge those three classes into one file named metastring.ts?

I am still studying the structure of the apache furry code base. I don't really know much but I think it's best to use the same file structure as the ones in the java code base , so that it will be easier for new developers to navigate through.

I prefer keeping javascript idioms. In fury python, all those three class will be put int one file named metastring.py. We won't get a fully consistent file layout across language.
@theweipeng How javascript organize such classes ?

I went through the https://www.conventionalcommits.org/en/v1.0.0/ but I still manage to fail the lint test.

Do we need to merge those three classes into one file named metastring.ts?

I am still studying the structure of the apache furry code base. I don't really know much but I think it's best to use the same file structure as the ones in the java code base , so that it will be easier for new developers to navigate through.

I prefer keeping javascript idioms. In fury python, all those three class will be put int one file named metastring.py. We won't get a fully consistent file layout across language.
@theweipeng How javascript organize such classes ?

We can put it in one file in fury typescript

Hello @theweipeng , I tried npm run lint --fix, but I can't seem to pass this  test . Please help me.

Could you try npm run lint-fix ? This command is declare in package.json. 'npm run lint --fix' means pass the param '--fix' to npm, but not to lint

Here is the test file MetaString.test.ts
import {describe, expect, test} from '@jest/globals';
import MetaString from './MetaString'; // Adjust the import path as necessary

describe('MetaString', () => {
  test('should encode and decode a simple string', () => {
    const input = "helloWorld_123";
    const encoded = MetaString.encode(input);
    const decoded = MetaString.decode(encoded);
    expect(decoded).toEqual(input);
  });

  test('should encode and decode a string with emojis', () => {
    const input = "helloWorld_123 💯";
    const encoded = MetaString.encode(input);
    const decoded = MetaString.decode(encoded);
    expect(decoded).toEqual(input);
  });

  test('should throw error for encoding string with .', () => {
    const input = "hello.World_123";
    expect(() => {
      MetaString.encode(input);
    }).toThrow("Unsupported character for encoding: .");
  });

  test('should throw error for encoding string with $', () => {
    const input = "hello$World_123";
    expect(() => {
      MetaString.encode(input);
    }).toThrow("Unsupported character for encoding: $");
  });

  test('should throw error for decoding string with .', () => {
    const input = "helloWorld_123";
    const encoded = MetaString.encode(input);
    encoded[0] = 0x2E; // Add a `.` character code point in the encoded data
    expect(() => {
      MetaString.decode(encoded);
    }).toThrow("Unsupported character for decoding: .");
  });

  test('should throw error for decoding string with $', () => {
    const input = "helloWorld_123";
    const encoded = MetaString.encode(input);
    encoded[0] = 0x24; // Add a `$` character code point in the encoded data
    expect(() => {
      MetaString.decode(encoded);
    }).toThrow("Unsupported character for decoding: $");
  });

  test('should encode and decode a string with various unicode characters', () => {
    const input = "helloWorld_123 💯 你好 мир";
    const encoded = MetaString.encode(input);
    const decoded = MetaString.decode(encoded);
    expect(decoded).toEqual(input);
  });
});

Please how do I initialize fury ?

Here is the test file MetaString.test.ts

import {describe, expect, test} from '@jest/globals';

import MetaString from './MetaString'; // Adjust the import path as necessary

describe('MetaString', () => {

  test('should encode and decode a simple string', () => {

    const input = "helloWorld_123";

    const encoded = MetaString.encode(input);

    const decoded = MetaString.decode(encoded);

    expect(decoded).toEqual(input);

  });

  test('should encode and decode a string with emojis', () => {

    const input = "helloWorld_123 💯";

    const encoded = MetaString.encode(input);

    const decoded = MetaString.decode(encoded);

    expect(decoded).toEqual(input);

  });

  test('should throw error for encoding string with .', () => {

    const input = "hello.World_123";

    expect(() => {

      MetaString.encode(input);

    }).toThrow("Unsupported character for encoding: .");

  });

  test('should throw error for encoding string with $', () => {

    const input = "hello$World_123";

    expect(() => {

      MetaString.encode(input);

    }).toThrow("Unsupported character for encoding: $");

  });

  test('should throw error for decoding string with .', () => {

    const input = "helloWorld_123";

    const encoded = MetaString.encode(input);

    encoded[0] = 0x2E; // Add a `.` character code point in the encoded data

    expect(() => {

      MetaString.decode(encoded);

    }).toThrow("Unsupported character for decoding: .");

  });

  test('should throw error for decoding string with $', () => {

    const input = "helloWorld_123";

    const encoded = MetaString.encode(input);

    encoded[0] = 0x24; // Add a `$` character code point in the encoded data

    expect(() => {

      MetaString.decode(encoded);

    }).toThrow("Unsupported character for decoding: $");

  });

  test('should encode and decode a string with various unicode characters', () => {

    const input = "helloWorld_123 💯 你好 мир";

    const encoded = MetaString.encode(input);

    const decoded = MetaString.decode(encoded);

    expect(decoded).toEqual(input);

  });

});

Please how do I initialize fury ?

Great work! You can now initialize a Fury instance by invoking 'new Fury()'. For examples of its usage, please search within the project. Note that the Compatible Mode is currently not supported, meaning that the MateString encoding and decoding functions are not being called anywhere in the system.

Please , how may I extend the BaseSerializerGenerator
this is my new MetaString.ts
import { TypeDescription } from "../description";
import { CodecBuilder } from "./builder";
import { BaseSerializerGenerator } from "./serializer";
import { CodegenRegistry } from "./router";
import { InternalSerializerType } from "../type";
import { Scope } from "./scope";

class MetaString extends BaseSerializerGenerator {
  description: TypeDescription;

  constructor(description: TypeDescription, builder: CodecBuilder, scope: Scope) {
    super(description, builder, scope);
    this.description = description;
  }

  // Encode function that handles all Unicode characters
  static writeStmt(accessor: string): Uint8Array {
    const encoder = new TextEncoder();
    return encoder.encode(accessor);
  }

  // Decoding function that handles all Unicode characters
  static readStmt(accessor: Uint8Array): string {
    const decoder = new TextDecoder();
    return decoder.decode(accessor);
  }
}

CodegenRegistry.register(InternalSerializerType.METASTRING, MetaString);

but  I get this error

Please , how may I extend the BaseSerializerGenerator
this is my new MetaString.ts
import { TypeDescription } from "../description";
import { CodecBuilder } from "./builder";
import { BaseSerializerGenerator } from "./serializer";
import { CodegenRegistry } from "./router";
import { InternalSerializerType } from "../type";
import { Scope } from "./scope";

class MetaString extends BaseSerializerGenerator {
  description: TypeDescription;

  constructor(description: TypeDescription, builder: CodecBuilder, scope: Scope) {
    super(description, builder, scope);
    this.description = description;
  }

  // Encode function that handles all Unicode characters
  static writeStmt(accessor: string): Uint8Array {
    const encoder = new TextEncoder();
    return encoder.encode(accessor);
  }

  // Decoding function that handles all Unicode characters
  static readStmt(accessor: Uint8Array): string {
    const decoder = new TextDecoder();
    return decoder.decode(accessor);
  }
}

CodegenRegistry.register(InternalSerializerType.METASTRING, MetaString);

but I get this error

BaseSerializerGenerator is used to generate code for types that users register using Type.xxxx(). MetaString is used to compute the meta information when we generate code for Type.Object. Therefore, you can invoke your function in gen/object.ts. Take a look at the other function computeStructHash, which is used to compute the hash of the type.MetaString works similarly to computeStructHash. MetaString only needs to be called once for each type during code generation, meaning that it is unnecessary to generate JavaScript code every time.

Hello @theweipeng , I have exported the metastring  class and called it  objects.ts , please have a look at my PR. Am I in the right direction ?

Hello @theweipeng , I have exported the metastring  class and called it  objects.ts , please have a look at my PR. Am I in the right direction ?

Yep, It is in the right direction.

@theweipeng  , please help me with a few hints on passing this job https://github.com/apache/fury/actions/runs/9747569516/job/26900584218?pr=1675

Hi, @theweipeng , Please have a look at my PR. I have been trying to adjust the object.test.ts files so as to pass the pipline tests  but i really don't know the writeStmt and readStmt functions  are tested in the objects.test.ts files. Please help me with a few hints.

Hi, @theweipeng , Please have a look at my PR. I have been trying to adjust the object.test.ts files so as to pass the pipline tests but i really don't know the writeStmt and readStmt functions are tested in the objects.test.ts files. Please help me with a few hints.

First, let me explain why we need to generate code, which is necessary for a general understanding.
Without generating code for each object type, the JavaScript would look like this:
 function writeObject(input, writer, description) {
     description.fields.forEach(field => {
         const binary = metaString.encode(field.name);
         writer.write(binary);
     });
     description.fields.forEach(field => {
         writeXxxx(input[field.name], writer, field.description);
     });
 }

In this pseudo-code, the function writeObject calls metaString.encode every time and iterating over description.fields is unnecessary. Performance is very important for Fury.
If we generate code for each object type, the JavaScript will look like this:
 const metaBinary = description.fields.map(field => metaString.encode(field.name)).join(/* code to join the binary data */);
 function writeObject(input, writer) {
     writer.write(metaBinary);
     writeString(input.b);
     writeBoolean(input.c);
 }

In this pseudo-code, metaBinary is called only once when writeObject is created, and we can avoid all overhead except for the necessary write buffer.
writeStmt is used to generate a code fragment that will be embedded by the function toSerializer, declared in BaseSerializerGenerator.
For example, a description like the one below will generate code when registerSerializer is called:
 const description = Type.object('example.foo', {
     b: Type.string(),
     c: Type.bool()
 });

 const fury = new Fury({ refTracking: true });
 const serializer = fury.registerSerializer(description).serializer;

The call chain will be generateSerializer(gen/index.ts) -> generate(gen/index.ts) -> generator.toSerializer(gen/index.ts) -> toSerializer(gen/serializer.ts) -> writeStmt(gen/object.ts). funcString declared in generate(gen/index.ts) store the final code.
writeStmt iterates through the properties defined in the description (in the example, they are b and c) and generates the code based on the property's description.
In this example, prop b will call writeStmt declared in gen/string.ts.
You can add a script for debugging like this:
https://github.com/apache/fury/assets/16490211/2c1f7b34-a4bd-4774-bc26-556b80a89c02
And click Debug:
https://github.com/apache/fury/assets/16490211/77d67ae8-042a-4ae1-b878-9de39777fb49
The test will pause at the breakpoint, allowing you to trace the call chain by debugging the code generator.
In your case, you should create a write instance for meta inside a closure, thus allowing direct writing of the meta binary.
For instance:
const metaBinary = this.scope.declare("metaBinary", `${metaString.encode(JSON.stringify(description))}`); // This line declares a variable inside a closure.
return `
${this.builder.writer.binary(metaBinary)};
`;

tagWriter is in a similar situation; this should be helpful to you:
https://github.com/apache/fury/assets/16490211/401e9f36-09c6-4204-9102-b115fac9631a

thank you for your guidance sir , the explanations saved me , please have a look at my PR.


Adding ability to override default enrollment period

closes #748

There are unsigned commits in the merge to bring this up to date with release-next. Closing to create a new branch and PR.


[BUG] A open stream with BlobOpenReadOptions(true) says that stream.canSeek true but says cannot seek on end

Describe the bug
A open stream with BlobOpenReadOptions(true) says that stream.canSeek true but says cannot seek on end. In my case, the stream is handed over to external libraries and on this point there is no possibility to check which seek is used.
Expected behavior
As the error message says that 'Cannot Seek with SeekOrigin.End on a growing blob or file.  Call Stream.Seek(Stream.Length, SeekOrigin.Begin) to get to the end of known data.'  The value of stream.canSeek should be false or Seek(0, SeekOrigin.End) should work like Seek(Stream.Lenght, SeekOrigin.Begin)
Actual behavior (include Exception or Stack Trace)
System.ArgumentException: 'Cannot Seek with SeekOrigin.End on a growing blob or file.  Call Stream.Seek(Stream.Length, SeekOrigin.Begin) to get to the end of known data. Arg_ParamName_Name'
To Reproduce
        [SkippableFact]
        public async void TestDecompressErrorForZip()
        {
            string connectionString = Constants.ConnectionStringTestBlobStorage;
            string fileName = $"anyfile{Guid.NewGuid()}.zip";
            BlobContainerClient blobContainerClient= new BlobContainerClient(connectionString, Constants.TestContainerName.Value);

            BlobClient blob = blobContainerClient.GetBlobClient(fileName);
            await blob.UploadAsync("anyfile.zip");

            using (var fileStreamErr = await blob.OpenReadAsync(new BlobOpenReadOptions(false)))
            {
                if (fileStreamErr.CanSeek)
                {
                    fileStreamErr.ReadByte();
                    fileStreamErr.Seek(0, SeekOrigin.Begin);
                    fileStreamErr.Seek(0, SeekOrigin.End);
                }
            }

            using (var fileStreamErr = await blob.OpenReadAsync(new BlobOpenReadOptions(true)))
            {
                if (fileStreamErr.CanSeek)
                {
                    fileStreamErr.ReadByte();
                    fileStreamErr.Seek(0, SeekOrigin.Begin);
                    // This should not fail, because CanSeek is true
                    fileStreamErr.Seek(0, SeekOrigin.End);
                }
            }
        }

Environment:

Azure.Storage.Blobs 12.9.1

I think having the end not be seekable might be by design since it's openRead. @seanmcc-msft could you chime in about this?
Not sure if we can also make canSeek return false either.. that's like a half breaking change. This seems like a weird usage to expect an OpenRead Stream to know where the end is when it's a growing stream.

This is by design.  AllowModifications=true means that the underlying blob could be growing, therefore it is not possible to seek to the guaranteed end of the blob.

Sorry, this isn't the behavior of other stream. As you can see in the example below FileStream jumps to the current end of the file when Seek(0, SeekOrigin.End) is called. And like in the example the size of the file is changed at any time.
The description of Stream.Seek also indicates this behavior

Use the CanSeek property to determine whether the current instance supports seeking.
...
Classes derived from Stream that support seeking must override this method to provide the functionality described above.

using var writeStream = new FileStream("test.bin", FileMode.Create, FileAccess.Write, FileShare.ReadWrite);
using var readStream = new FileStream("test.bin", FileMode.Open, FileAccess.Read, FileShare.ReadWrite);

writeStream.Write(new byte[100], 0, 100);
writeStream.Flush();
Console.WriteLine($"Write Position: {writeStream.Position}, Length: {writeStream.Length}"); // => 100, 100
Console.WriteLine($"Read Position: {readStream.Position}, Length: {readStream.Length}"); // => 0, 100

readStream.Seek(0, SeekOrigin.End);
Console.WriteLine($"Read Position: {readStream.Position}, Length: {readStream.Length}"); // => 100, 100

writeStream.Write(new byte[100], 0, 100);
writeStream.Flush();
Console.WriteLine($"Write Position: {writeStream.Position}, Length: {writeStream.Length}"); // => 200, 200
Console.WriteLine($"Read Position: {readStream.Position}, Length: {readStream.Length}"); // => 100, 200

Unfortunately, changing this behavior may be considered a breaking change, which we are not allowed to make.  Other users may have taken a dependency on this behavior.

You anyone is expect a exception to be thrown when he calls a method which says "you are not allowed to do this". I understand your point of backward compatibility but think it's only academical in this case because you remove a "fobiden" sign. Every one who expect the exception won't call this method.


mini.sessions cannot create a default sessions directory under windows

Contributing guidelines

[X] I have read CONTRIBUTING.md
[X] I have read CODE_OF_CONDUCT.md
[X] I have updated 'mini.nvim' to latest version

Module(s)
mini.sessions
Description
The default mini.sessions' path is incorrect in a way that at startup mini.sessions keeps erroring
(mini.sessions) "C:\\\\Users\\\\B4mbus\\\\AppData\\\\Local\\\\nvim-data\\\\session" is not a directory path

Changing it manually to vim.fn.stdpath('data') .. '\\sessions\\ works and it doesn't display that error anymore.
My understanding is that on windows to create the directory one must append \\ at the end. I'm willing to look into that, just creating that issue for organization purposes.
(related to #133, but I'm splitting that issue into two)
Neovim version
0.8.0-dev-922-g42e9fe7d9
Steps to reproduce

nvim -nu minimal.lua
Contents of minimal.lua:

require('mini.sessions').setup({})

Expected behavior
mini.session properly creating, recognizing and using the default directory.
Actual behavior

The 'mini.sessions' now automatically creates directory if it is not present.
Could you, please, update and see if the issue still persists?

I was just able to verify on Windows machine that 'mini.sessions' works as expected after initial set up. Itcreated global default directory and was able to write/read/delete a session there.
So closing this now.If you still experience this, feel free to reopen.


Proposed User Story: Query what codebases uses a subset of gems

I'd like to be able to use Quimby to see what codebases (that are Ruby) use a subset or selection of gems.
Example: Tell me all the repositories that currently use dor-services. Versions too could be helpful, but I'm primarily interested in who is using what gems or libraries.
This can probably be a query completed through Gemfile lookups / searching.
This is a low priority.

we could potentially look into functionality via the bundler-audit or bundler gem that provides Gemfile.lock parsing classes

Gemnasium also knows this: https://gemnasium.com/gems/dor-services

so maybe in the spirit of quimby handing off queries to other services that are already/better suited to handle them, we just recommend folks needing this go to gemnasium.

We can query that data via the gemnasium API and load it into a table, then do queries against that.  That could provide the search usefulness, but then leaving the UI of actually displaying that information prettily as a link into gemnasium if people want to drill down more.
A lot of the reason why I wanted to look at this was to rethink a little bit from something @eefahy said in the retrospective about using bundler-audit with security parsing as a whole.  The advantage to having bundler do the checks for this and for security problems in general is that the developers know for sure that the thing they'll use to distribute a package is happy with it.  That does have some appeal.
On the other hand, gemnasium actually provides an API rather than making us figure out how to best run this process and get back the info ourselves.  That's a major thing for ease of setting up the app and debugging it in the future.
Having more time to think it through, I think I'd recommend that for now we continue using gemnasium for the alerts that we already are, and that I edit that process to also pull in the names of all the gem dependancies for us to expose as a filter.  The thought of a gemnasium/bundler-audit mismatch is only theoretical and they do (IIRC on gemnasium) use the same database to determine this.  So we can do the simple thing for now and only spend time on complicating it if it does turn out that in practice there's a good benefit to running the parsing directly ourselves.


docs: differentiate core vs js

This PR attempts to better differentiate when you need autocomplete-js and autocomplete-core in the getting started page.
I wonder if it may be best to actually just remove mention of core here, since I imagine very few users will need this, and it could cause confusion for most users.
If we want to keep it, could you help had clear use cases to the second paragraph @francoischalifour ? My current language (unable to create the experience you want) is still too vague.

Closing this since we made it clearer on the official docs.


Feature/configurable migrations directory

Closes #7

:tada: This PR is included in version 2.3.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Creating mp4/m4a file

Hi
I'm trying to create an m4a file, but have been unable to find the necessary method. So far I have an AAC file, but need to wrap it in the M4A container. Is this supported in NAudio and if so are there any samples or docs relating to this.
Cheers
Keith

NAudio basically exposes the ability to use the Media Foundation API in C#. As I understand it, when you create a Media Foundation "sink writer" it should choose an appropriate container based on the file extension you pass in. So for AAC you can use AAC or MP4/M4A, and it should create the correct format. Note that not all containers are supported on all versions of Windows. For example, on Windows 7 you can't make AAC files, but you can make MP4 with AAC in it.


Installing diff_cover adds tests to site-packages

Installing this package via pip adds the tests folder as a package in the virtual environments site-packages. I.e.:
> ls .venv/lib/python3.9/site-packages/tests
fixtures/    __pycache__/                      test_clover_violations_reporter.py  test_diff_cover_tool.py    test_git_diff.py     test_java_violations_reporter.py  test_util.py
helpers.py   snippet_list_unicode.html         test_config_parser.py               test_diff_quality_main.py  test_git_path.py     test_report_generator.py          test_violations_reporter.py
__init__.py  test_clover_violations_reporter/  test_diff_cover_main.py             test_diff_reporter.py      test_integration.py  test_snippets.py

This means that it is not possible to use the name tests as a module when diff_cover is installed. diff_cover's tests now precedes local modules in the import path. I don't know if this is intentional or not, but it is fairly impractical, as tests is a very common module name in most projects.
Not sure if this is related to https://github.com/Bachmann1234/diff_cover/issues/209

Oh jeez. I'll fix this. Sorry for the trouble

@eirki https://pypi.org/manage/project/diff-cover/release/6.3.5/ should fix this. I went ahead and yanked 6.3.4. Sorry again for the trouble

@eirki https://pypi.org/manage/project/diff-cover/release/6.3.5/ should fix this. I went ahead and yanked 6.3.4. Sorry again for the trouble

Nothing to apologize for, thanks for the quick fix!


riot-desktop: use yarn2nix-moretea

Motivation for this change
Supersedes #67514 and adds a dependency update script.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Notify maintainers
cc @worldofpeace

It should not be necessary to commit yarn.lock to nixpkgs.

Fix in https://github.com/NixOS/nixpkgs/pull/67534

Closing in favour of #67534.


"out of memory" on azure pipelines

monaco-editor version: 0.15.6
OS: Azure pipelines
Steps or JS usage snippet reproducing the issue:
We have a project with webpack which works fine on our machine. But we can not make a build on azure pipelines. It returns out of memory.
We used monaco-editor-webpack-plugin. https://github.com/Microsoft/monaco-editor-webpack-plugin
Do you have a recommendation?

Isn't this more of an issue for webpack? I don't think we are doing anything memory intensive in our plugin. Perhaps other folks in that community have tips for how to reduce memory usage in webpack?

@alexandrudima
The only difference with working one and not working one is monaco-editor with webpack-plugin.
That is why i opened issue here.
Thank you.

I think you might have similar issues with webpack if you include any other multi-megabyte library, i.e. I think it has to do with the size, not with the content.

You are right on the point. It is about the library size. But other people might have the same problem. I though you could give some tips.

Yes, other people might have the same problem, but I think those people are far more likely to be found on the webpack issue tracker / forum / etc.
I am sorry, I really believe the project that allocates memory should know tweaks on how to reduce memory allocation. In this case, when you invoke webpack, it is webpack that allocates memory and not the source code that is being webpacked.
One thing I would try is to disable minification, AFAIK minification has a huge memory footprint regardless of using webpack or not. Perhaps you can webpack everything and then minify each file in a separate step.

Thank you very much. That is more than i could expect.


When executing from a lambda function, the SNS subject is too long when using ARNs

thank you great tool.
When executing from a lambda function, the SNS subject is too long when using ARNs
for example:
Tracer: arn:aws:ecs:ap-northeast-1:012345678901:task/dev-main/0123456789abcdef0123456789abcdef on arn:aws:ecs:ap-northeast-1:012345678901:cluster/main
2022-03-23T03:34:40.757Z	TASK	Created
2022-03-23T03:34:44.431Z	TASK	Connected
2022-03-23T03:34:58.254Z	TASK	Pull started
2022-03-23T03:35:18.965Z	TASK	Pull stopped
2022-03-23T03:35:37.090Z	TASK	Started
2022-03-23T03:41:15.397Z	TASK	Execution stopped
2022-03-23T03:41:25.441Z	TASK	Stopping
2022-03-23T03:41:25.441Z	TASK	StoppedReason:Essential container in task exited
2022-03-23T03:41:25.441Z	TASK	StoppedCode:EssentialContainerExited
2022-03-23T03:41:38.629Z	TASK	Stopped
2022-03-23T03:44:32.977Z	CONTAINER:hoge	LastStatus:STOPPED HealthStatus:UNKNOWN (exit code: 0)
2022-03-23T03:44:32.977Z	TASK	LastStatus:STOPPED
InvalidParameter: Invalid parameter: Subject

from aws cli help message:

       --subject (string)
          Optional parameter to be used as the "Subject" line when the message
          is  delivered  to email endpoints. This field will also be included,
          if present, in the standard JSON messages delivered  to  other  end-
          points.

          Constraints:  Subjects must be ASCII text that begins with a letter,
          number, or punctuation mark; must not include line breaks or control
          characters; and must be less than 100 characters long.

Sending SNS fails because the subject is too long.
Therefore, we have made two changes.

cluster name and task ID are extracted from ARN in the Lambda handler.
if the Subject is longer than 100 characters, omit the end

thank you!


Implemented dynamic tty column width resolution

I started using wunderline today and found that task title gets truncated when you have tasks title running for more than 60 characters or so, since most of my tasks are running past this limit I couldn't pretty much list my daily TODO tasks effectively.
An example:
Current implementation

What I am proposing

Thanks for this @purinda, just checking it out on my machine quickly :)

Published as v4.1.5


LINQ filtering methods

Hello,
I'm having some issues with the LINQ implementation, specifically the .Where() method. I'm trying to filter some data using methods like:
.Where(x => x.Contains("string"))
or
.Where(x => x.Id.Equals(id))
Here's an example:
var q = await arango.Query<AnotherTestPerson>("TestDB").Where(x => x.Fullname.Contains("D")).FirstOrDefaultAsync();
Is there any possibility for getting support for these methods in the future? For now I have to convert the arango IQueryable to a list, then cast it again as a IQuearyable to access those methods.

Just rewrite them with
.Where(x => Aql.Contains(x.Name, "D"))

or
.Where(x => x.Id == id)

Supporting the other notations should be possible though.

So calling methods normally in LINQ need to be done via the Aql functions? Or it's just the ones I listed here? I haven't tested other methods.

Yeah for now use the Aql. functions as they are bound to work natively - the ones with params are broken though
need to reqrite them with 1..5 named params or so
Just noticed even a basic list.Contains(y.Id)
needs to be written as Aql.Position(list, x.Prop)
if any aql function is missing you can write one yourself with
public partial class AqlCustom
{
        private static Exception E => new NotImplementedException();

        [AqlFunction("FNV64")]
        public static string Fnv64(object value)
        {
            throw E;
        }
}

Hi @coronabytes ,
I'm working together with @Mrjuanblack on a project where we use this project to integrate with Arango. Thank you for your feedback so far.
The problem we are facing is that we need to integrate with a number of clients through a LINQ abstraction. They are currently depending on this abstraction to make things work with other databases and we should be able to just swap the implementation to one supporting Arango without breaking the interface. Because of that, it is not feasible to rely on Aql helpers -- methods like 'Contains' should work like in any other LINQ implementation.
What is the underlying reason for the current implementation through Aql helpers? And what kind of work would need to be done to support things like '.Contains' directly? We are happy to contribute to the project to get this to work but need some pointers for where to start.
Looking forward to working with you on this.

Well the LINQ code is basically just a fork of the first Arango .NET driver. (https://github.com/ra0o0f/arangoclient.net)
Relinq should be able to quite easily support String. Math. and Date. functions,
if its GroupBy you are looking for thats gota be a challenge, but contributions are welcome,
I'll try to add String.contains and then the rest is just copy pasta.

Ok got you a starting point
https://github.com/coronabytes/dotnet-arangodb/tree/LinqMethods
the tricky part was finding out  how to move the <x.Name>.Contains inside the Arguments, but thats seems to work,
Please do not map StartsWith, EndsWith, Contains with LIKE Operator, because of % injection, use the native aql methods
also try x.Name.ToLower().Trim().Contains() and so on

@coronabytes awesome. We'll start working on this on a fork (after the holidays probably).


Should I run the script before or after user creation?

Hi @farag2 ! Hope you're doing well. Thanks for this amazing script.
My dad uses sometimes my pc so I have to create an exclusive user for him.
Should I run your script before or after doing that?
Best regards!

Hi! After user creation of course. Most of the settings are being applied for current user.

Hi! After user creation of course. Most of the settings are being applied for current user.

Thank you so much!

Thank you so much!


After migrating from 1.6.210 to 2.5.207, existing method not working

Configuration:

Web browser and its version: Chrome 85
Operating system and its version: Windows 10
PDF.js version: 2.5.207
Is a browser extension: no

Migrating from 1.6.210 to 2.5.207
Previous code
 var loadingTask = PDFJS.getDocument(pageSrcNormal);
loadingTask.promise.then(function (pdf) {
       getPagePdf(omniview, pdf, 1);
}, function (reason) {
     // PDF loading error
      console.error(reason);
      leaflet.spin(false);
});

Current Code:
 var loadingTask = pdfjsLib.getDocument(pageSrcNormal);
loadingTask.promise.then(function (pdf) {
       getPagePdf(omniview, pdf, 1);
}, function (reason) {
     // PDF loading error
      console.error(reason);
      leaflet.spin(false);
});

pageSrcNormal is URL of dotnet api, which will return the pdf file dynamically
What is the expected behavior?
Document's preview should render on screen after getting data from API
What went wrong?
The API is not getting hit

Note: with previous version the method is working as expected

The error message in your screen-shot clearly points out bugs in your environment, and not in the PDF.js library, and as mentioned in the error message you've added[1] two methods (min and max) to the Array.prototype in such a way that they become enumerable.
Having methods on built-in primitives, such as e.g. Array, be enumerable will lead to (strange font-related) breakage elsewhere in the PDF.js code-base, which is why the library will now refuse to load in those circumstances.

[1] It's also possible that you're using a JavaScript framework which is responsible for this error, however we cannot really help with that (even more so, since no runnable test-case was provided).


Pipeline: Release notes generation is not working in some cases

When we did the 3.0.0 Release the release notes were empty.
Todos:

Analyse root cause (Commitzen, Pipeline Issue, perhaps only with breaking change commits?)
Fix it.

Additional problem related to this:
When a PR is merged and no version bump will be done the build fails (and it shouldn't),
e.g. https://github.com/boschglobal/doxysphinx/commit/7bd33b2098c1e42ff781fa6b3d6be8f7dd5d61d9


Update Dockerfile

Remove one abstraction layer
Make user nonroot
Bump golang version to 1.14

Codecov Report

Merging #18 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #18   +/-   ##
=======================================
  Coverage   16.10%   16.10%
=======================================
  Files           5        5
  Lines         298      298
=======================================
  Hits           48       48
  Misses        249      249
  Partials        1        1

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6f76521...6882e97. Read the comment docs.

LGTM, thank you!


Redundant information in LeaderId::node_id

Currently, LeaderId contains not only term, which is sufficient to identify the leader's term and thus also the validity of log entries, but it also contains node_id of the leader.
This information is redundant, but nevertheless always transported via LogId (which subsumes LeaderId). This leads to unnecessary complexity, since either the node ID of the leader must be logged or otherwise recovered at log replay time.
The only location in code where LeaderId::node_id is evaluated seems to be in Engine::update_progress(), where it would likely suffice to compare the term (the same term cannot be used by two different leaders).
There is a TODO in leader_id.rs already. Most likely, it would be sufficient to simply drop node_id from this structure.
In any case, removing the comparison for testing purposes doesn't have any effect on test outcome - all green.
diff --git a/openraft/src/engine/handler/replication_handler.rs b/openraft/src/engine/handler/replication_handler.rs
index 4d0d86c..165a630 100644
--- a/openraft/src/engine/handler/replication_handler.rs
+++ b/openraft/src/engine/handler/replication_handler.rs
@@ -86,7 +86,7 @@ where
     pub(crate) fn try_commit_granted(&mut self, granted: Option<LogId<NID>>) {
         // Only when the log id is proposed by current leader, it is committed.
         if let Some(c) = granted {
-            if c.leader_id.term != self.state.vote.term || c.leader_id.node_id != self.state.vote.node_id {
+            if c.leader_id.term != self.state.vote.term {
                 return;
             }
         }

My suggestion is to remove the node_id member, since it's basically unused and not needed. Going further, maybe the entire LeaderId structure can be simply replaced by term?
Or is there some error in my thinking?
Thanks.

I added a node_id to LeaderId because LeaderId(term, node_id) reduces conflict during an election:
Given 3 nodes, if all of them start to elect at the same time, with the same term, without node_id, it is very likely there is no leader established in this term:

a voted for itself and sent vote requests to b, c;
b voted for itself and sent vote requests to a, c;
c voted for itself and sent vote requests to a, b;

Because they all have already voted for themselves, they just reject every received vote request from other nodes. This is the way the standard raft does
But with node_id, finally there will be a leader elected in this term. Because LeaderId(term=1, node_id=2) > LeaderId(term=1, node_id=1), the node with greatest node_id will finally succeed.

In any case, removing the comparison for testing purposes doesn't have any effect on test outcome - all green.

It has to compare node_id too here, because there might have been multiple leaders in the same term, e.g., LeaderId(term=3,node_id=a) then a new LeaderId(term=3,node_id=b) established. In this case, a log of LeaderId(term=3,node_id=a) being replicated by LeaderId(term=3,node_id=b) does not imply the log is committed.
It looks I have to add some tests for such a case:(

The only location in code where LeaderId::node_id is evaluated seems to be in Engine::update_progress(), where it would likely suffice to compare the term (the same term cannot be used by two different leaders).

This is the only location where it is explicitly used. When handling a vote request, a node just compares votes with req.vote >= self.vote, this is the most important usage of it.

Hm, I still don't get it. The standard Raft uses only term, but no node_id for its election and it works correctly. Raft guarantees by its election process that there cannot be two leaders elected for the same term. So how come there are two leaders with the same term in openraft?
In the first case you described (all start voting for self at the exact time), there would be no leader elected for this term, which would trigger another election with a new term. The likelihood that even the second election again occurs at exact the same time on all nodes (and fails too) is virtually zero.
When I understand you correctly, you only store node_id to speed up the election? If so, I don't think it makes sense to optimize the "bad" case (new election occurring once in a while) by penalizing the "good" case all the time (e.g., additional info to be logged; occurring all the time).
The vote handling compares log ID (i.e., term + log index). That should be sufficient, since the same term cannot be used by two different nodes. I don't see how the node_id should influence anything there.

Hm, I still don't get it. The standard Raft uses only term, but no node_id for its election and it works correctly. Raft guarantees by its election process that there cannot be two leaders elected for the same term. So how come there are two leaders with the same term in openraft?

In standard raft, a leader is not just identified by term. A leader is identified by a tuple LeaderId: (term, voted_for), e.g., a node has voted for node-1 in term 3 has its LeaderId stored as (term=3, voted_for=1).

The LeaderId in standard raft is a partial-order value, because voted_for=1 !> voted_for=2 and voted_for=1 !< voted_for=2. I had a slide explaining the conflict when candidate y in term 5 tries sending vote request to another node in term 5:

And this partial-order LeaderId allows only one leader to be granted in every term in the standard raft, term, without voted_for can be used as a leader identifier.
In openraft LeaderId(term, node_id) is a total-order value, i.e., LeaderId(term=3, node_id=2) > LeaderId(term=3,node_id=1), which makes it possible to have multiple leaders in one term. But still, at any time, there is only one leader is granted by a quorum. E.g., LeaderId(term=3,node_id=1) is granted by node-1 and node-2(3 nodes cluster), then LeaderId(term=3,node_id=2) is granted by node-1 and node-2. Because LeaderId is totally ordered, a node can grant LeaderId(term=3,node_id=2) even when it has already granted LeaderId(term=3,node_id=1).

In the first case you described (all start voting for self at the exact time), there would be no leader elected for this term, which would trigger another election with a new term. The likelihood that even the second election again occurs at exact the same time on all nodes (and fails too) is virtually zero.

Right, the chance there is another round of conflict is rare. But still, it takes two rounds of election. The time gap between these two rounds will be considered service downtime(about 2 RTT).

When I understand you correctly, you only store node_id to speed up the election?

Generally speaking, yes:). And it simplifies the logic. Adding another field will save several lines of code that handle term.

If so, I don't think it makes sense to optimize the "bad" case (new election occurring once in a while) by penalizing the "good" case all the time (e.g., additional info to be logged; occurring all the time).

Hm... to me,  with or without a node_id in a log entry, the storage or transport efficiency can be well optimized when needed, e.g., extracting term, node_id and storing them separately. But the election conflict can not be solved without node_id in the LeaderId.

Wow, what a long explanation :-), thanks.
I was imprecise, sorry. Of course, (one) voted-for node ID needs to be stored and the node_id is sent as part of the Vote message.
So we can conclude that the node_id storage is for speeding up the election. I can live with that, I just have to find a way how to optimize log storage that we don't have to store the additional node_id information for each log entry unnecessarily. Maybe something like not storing the term/node_id per log entry at all and rather generate "switch" log entries when there is a change.
Alternatively, looking at the code again, it seems like Entry could use a simpler LogId only containing term and index, without the node_id. The node_id seems to be only relevant for replication reply, where it can be easily computed based on the last voted_for for Matching. For Conflict, it can be ignored, since it's not used. Of course, the default implementation of RaftLogReader would be more complex/impossible and quite a few changes would be needed elsewhere...
I'll probably go with the above mentioned storage optimization in the log ("switch" entries) or we'll store our 16B node IDs for each log entry anyway.

Alternatively, looking at the code again, it seems like Entry could use a simpler LogId only containing term and index, without the node_id. The node_id seems to be only relevant for replication reply, where it can be easily computed based on the last voted_for for Matching. For Conflict, it can be ignored, since it's not used. Of course, the default implementation of RaftLogReader would be more complex/impossible and quite a few changes would be needed elsewhere...

LogId is also used on a follower to determine if a log entry is identical to the follower's local log entry at the same index, when a follower receives append-entries request.
I have a LogIdList structure that contains every log id from which a new LeaderId starts:
https://github.com/datafuselabs/openraft/blob/main/openraft/src/engine/log_id_list.rs
In fact, in my opinion, such a structure reflects more precisely the data structure in raft.
Logs should be a 2 dimension list entry = logs[leader_id][index].
May be I can introduce a feature to disable/enable to contain a node-id in LogId. 🤔

I'll probably go with the above mentioned storage optimization in the log ("switch" entries) or we'll store our 16B node IDs for each log entry anyway.

I did not know you have such a big node-id :(
Is it possible using a u32 node-id and store the 16B real id in a Node?
Membership stores every NodeId and Node for applications:
https://github.com/datafuselabs/openraft/blob/e31b0f9c8601a995b69c5b8200f62af286e588bb/openraft/src/membership/membership.rs#L73-L87

LogId is also used on a follower to determine if a log entry is identical to the follower's local log entry at the same index, when a follower receives append-entries request.

Sure, but considering that Raft doesn't allow to have two leaders for the same term, the term comparison must be a sufficient condition - no need for node_id there.

May be I can introduce a feature to disable/enable to contain a node-id in LogId. 🤔

I wouldn't complicate the API and core algorithms. Either-or, I'd say.
We have plenty of optimization potential elsewhere (e.g., get rid of Vec<T> and similar on the API, replacing them with impl IntoIterator<T> or similar to prevent unnecessary memory allocation, where not required, etc.).

I did not know you have such a big node-id :( Is it possible using a u32 node-id and store the 16B real id in a Node?

Indeed, I'm thinking about doing exactly that. So far, we didn't use the Node, but maybe it's a better way; simply store Raft node index in the node_id and use Node to store the actual node ID (we use 16B GUIDs; that's a long story). That would actually make some other operations simpler (restoring a consensus domain replica in a catastrophic scenario on a completely unrelated node w/o the need to "renumber" everything).


MalmoPython ImportError: DLL load failed: The specified module could not be found

MalmoPython cannot be imported, although there is MalmoPython lib in the Python_Example directory.
I searched for other cases, but they didn't help.
I'm using Windows10, Python2.7 64bit, and installed Malmo-0.17.0-Windows-64bit.
Inserting current path mentionded in #166 also didn't work.
import os, sys
sys.path.insert(0, os.getcwd())
import MalmoPython
...
Please help!
C:\Users\fujim\Documents\malmo\Malmo-0.17.0-Windows-64bit\Python_Examples>python run_mission.py
Traceback (most recent call last):
File "run_mission.py", line 22, in
import MalmoPython
ImportError: DLL load failed: The specified module could not be found
C:\Users\fujim\Documents\malmo\Malmo-0.17.0-Windows-64bit\Python_Examples>python
Python 2.7.12 (v2.7.12:d33e0cf91556, Jun 27 2016, 15:24:40) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

Sorry I solved it. Just forgetting to install codesynthesis.
#156 helped.


setup-ruby-path: Ensure that Ruby is installed

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same change?
[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[ ] Have you written new tests for your changes? Here's an example.
[x] Have you successfully run brew tests with your changes locally?

Fix the error:
/usr/local/Homebrew/Library/Homebrew/brew.sh: line 372: exec: : not found

Previously when no Ruby is installed, the vendored Ruby would be installed.
Now when no Ruby is installed, the above error message is printed.
I believe the change of behaviour occurred here:
https://github.com/Homebrew/brew/commit/b2b413165f3553c48c8e060dd97ccde9c13cae80#diff-2012e7e4fc656781b9d32ead2909ee8bR43
This PR restores the previous behaviour. When no Ruby is installed, the vendored Ruby will be installed.

Previously when no Ruby is installed, the vendored Ruby would be installed.

Just trying to understand this: is that if there's no Ruby found even in e.g. /usr/bin i.e. this is a Linux only bug? Happy to merge regardless but just want to make sure I understand.

Yes, correct. This bug only affects Linux, since macOS always has /usr/bin/ruby available.

Thanks again @sjackman!

My pleasure. Thanks again for merging, Mike.


New option: closingTagOnNewLine

Fix for #67 .
If this option is true, multiline tags will have the closing tag on its own line.
Instead of:
<div
  attr1='foo'
  attr2='bar'>
  A div.
</div>

You get:
<div
  attr1='foo'
  attr2='bar'
>
  A div.
</div>

This is not thoroughly tested, but works in my sample cases.

Hi @dreel, thanks for the PR. It seems the PR failed the test. Ideally this should be easy to fix by running the beautification of the files with npm run beautify.
Will take some time tomorrow to review and merge. Would you mind adding a simple unit test for this case? check the unit tests folder to see how the tests are done. Thanks in advance.

Hi @royriojas , I also found an edge case where this doesn't work as expected (a self-closing multiline tag, it should carry the /> instead of just the > to the next line). I will fix try to take a look at adding a unit test and fixing that case over the weekend.

hi @dreel, I really appreciate you take the time to fix that. Thank you man.

Fix and unit test is in, and I had to fix bug-34.js as well, which I think was unrelated to my changes. I'm guessing it was an update through npm that caused the tests to start complaining about the collision on the export of some.

nice @dreel will merge


Webassembly support?

Does reqwest work with wasm-unknown-unknown target, e.g. in a yew app to do ajax requests?

I am starting work on this. My thought is actually to simply have reqwest hand everything off to the fetch API. However that should be what projects depending on reqwest need if they have reqwest as a dependency (Which is my case).

Hey, I really support this.
It would be awesome to have the same code work on wasm and regular Rust.
That way you could reuse, eg, API clients on the frontend and backend.
As hyper wouldn't work in the browser anyway, my idea was also to use fetch under the hood and just provide the same API.
@corbinu  did you get anywhere with your implementation?
Otherwise I might give it a go.

@theduke It was a bit harder than I had hoped and I work in eCommerce so November was basically a goner for me as far as open source is concerned. I really do want to get back to this though so would be very happy if you wanted to collaborate.

@seanmonstar is this something that's at all reasonable for you or do you see this as definitely out of scope for reqwest?

@theduke the idea sounds reasonable, but I don't know if in practice it's an incredible burden to support all of reqwest's features using browser APIs or not. I haven't look into it at all.

@seanmonstar If this is not something your interested in I understand. However I do think that reqwest would actually probably get a lot of traffic on npm if it were available as a WebAssembly module.
Also my use case is I am porting over a DB driver I have in TypeScript to Rust. This module currently uses Wreck as an HTTP library. I want to be able to use the same driver both for new Rust projects and my existing TypeScript apps. So the driver needs to be able to compile into WebAssembly. So I need an HTTP client that can compile also. I have a feeling this will be a very common use case.
For example front end libraries in Rust are already popping up. They would probably love to use reqwest rather than calling out to something like Axios.
@theduke When I had started I actually got thinking it might be better to create a new fork and strip out all the hyper parts but keep a reqwest compatible API. Then look at merging the projects back together after I got it working. What are your thoughts?

@seanmonstar I am wondering your thoughts also on a fork that would have the intention of merging back in once async/await futures are stable?
Also I think best first steps would be to migrate to Rust 2018 and Futures 0.3 Is it alright if I open PRs for those?

Switching to the new edition would require the newest compiler, and I've generally tried as best as I can (sometimes dependencies use new features in patches) to not require new compiler in patch versions.
We also can't adopt Futures 0.3 yet since it requires a nightly compiler.

Ok totally makes sense.
Sorry I should have been clear I was just thinking the async module since that was nightly.
So that makes me more apt to take this route. How would you feel about a fork that is meant to run only on nightly for now that way it uses all the new stuff but it keeps the API so once all of the things are stable could be merged back in?

This would be great! I'm working on a wasm-bindgen-based frontend framework, and am looking for a way to make http requests in the browser. Have been wrestling with the web_sys fetch API, trying to make a high-level API wrapping it... seems similar to reqwest's lane.

@David-OConnor I have started work on a forked lib for this I am thinking of it as maybe a future vision for reqwest as I totally understand @seanmonstar 's desire for maintaining stability for current users. I will be publishing the repo in the next few days as work allows if you want to participate. I am also happy to add you @theduke. I already made it async only and migrated to Rust 2018. The plan for now is to make it work in nightly rust + WASM in the browser as a replacement for axios + WASM in node as a replacement for Wreck

WASM doesn't require anything unstable; the wasm32-unknown-unknown target now works on stable rust.

No it's all perfectly possible on stable. It just needs futures and wasm_bindgend_futures.
The browser event loop takes care of the rest.
I've also done a prototype and what I've noticed:
you basically can share almost no code.
I think the most sensible thing to do is to just add a new module with it's own ClientBuilder and Client which remain API comptatible as much as possible.
The feature dance is a bit challenging to get right in a clean fashion but doable.
And of course a lot of features aren't implementable (custom connectors, tls settings, timeout settings, ...).
Otherwise it's possible to get to a very similar API.
I guess we'll need to see if it's worthwhile to actually include it in reqwest.

Hey all, I just wanted to share some thoughts that may widen the scope of the issue. WebAssembly can target nodejs as well as the browser. If this library goes the route of supporting webassembly, it would be awesome if it were usable in nodejs as well.
I specifically like the idea of writing node libraries that can take advantage of rust when making http calls (scraper libraries come to mind right away)

Any update on this? I'm trying to port speedtest-rs to Wasmer and as stated at https://github.com/nelsonjchen/speedtest-rs/issues/20#issuecomment-515099642, I've found that reqwest fails to compile to wasm32-wasi on net2-rs... Since Wasmer is a posix-like WebAssembly environment instead of a browser-like, is there any clue about how I can fix that?

WASI doesn't support asynchronous sockets, so it probably won't get support in mio. Which means the whole reqest/hyper/tokio/mio stack won't work there sadly.

Being a new ABI, it's strange it doesn't support asynchronous sockets, in fact I would not have added synchronous ones due to performance... But it's true that (fake) asynchronous APIs can be build on top of synchronous ones, so maybe it can get added some time in the future.

I'm not familiar with all the details of WASI, but if there's native support for xhr/fetch, then I imagine reqwest can conditionally use those instead of hyper.

Not xhr/fetch like since WASI API is more like syscalls, but it has support for Berkeley sockets.
The thing is that in fact net2-rs already have wasi support, so I don't know why it crash... :-/ It's easy to reproduce, just only exec cargo build --target=wasm32-wasi in reqwest source code and you'll get the error at compile time.

I'm not familiar with all the details of WASI, but if there's native support for xhr/fetch, then I imagine reqwest can conditionally use those instead of hyper

AFAIK fetch is provided by stdweb.

It's important to distinguish between targets.
If you want to run wasm in the browser, you can use wasm-bindgen/web-sys and use fetch or good old XMLHttpRequest.
wasi is a different target that is for non-browser environments. All you get there are some low level syscalls including synchronous sockets. (WASI might get async sockets in the future, but I don't think it's currently worked on).

#630 sees a WIP start for adding wasm32 support. Various things don't work, and I don't typically do things with WASM, so if you want to comment or help out, the more the merrier :D

I've merged #630 which supports wasm in the browser. I'll open new issues for things still missing.


[BUG] State not abbreviating in /astro-data

Describe the bug

The state on the /astro-data page is not getting abbreviated if it’s a U.S. state.

Steps To Reproduce

Go through the steps to get to the /astro-data page
Observed the location just under the header

Expected behavior
If the location is within a U.S. state, the state should be abbreviated.

Latest PR fixes this issue. Just make sure the browser hasn’t cached the previous version.


chore: add compatibility with Android Gradle Plugin 8.0.

Issue #120

Thanks for the PR. Would you be able to:

Share any docs around these changes?
Leave the changelog and version upgrade changes out in this PR? I'll do that afterwards

Added link to Issue related to this change.


Add helper methods to e2e gridTestUtils.

Add getGrid, getRow, and selectRow helper methods to utils file.
Modified existing methods to use these new helper methods where
appropriate. In reference to issue #2693.

I used this file to run my project's e2e tests (which passed), but when attempting to run npm install to run the e2e tests for ng-grid, version errors were thrown for "glob".

I looked at the details of the build failure, but it doesn't look like a failure do to my code... unless I'm not interpreting the failure correctly.

Looks fine, I agree the fail isn't your code.  Merging.


Support console: streamline the editing a user's responsible body

Context
The support console allows removing and adding new schools for users of the system.
Changes proposed in this pull request
Apply the same interaction patterns for:

adding
updating
removing a user's responsible body

as are present in the school-user-facing parts of the app.
Guidance to review
Setting a responsible body when there wasn't one before
Before:

After:

Removing a responsible body and moving to a new responsible body
Before:

After:

@aldavidson good call: I think I've fixed it in 3e43102:

@aldavidson good call: I think I've fixed it in 3e43102:


Optimize TIFF converter data and metadata reads

Combines data and metadata extraction to avoid multiple file reads. This should help speedup TIFF conversion. However, it doesn't appear, on test files, that this matters. In either case, this code is cleaner and more maintainable.

Looks good now. Will be in the next release.


Search feature when using workspace is searching folders that should be ignored

Steps to Reproduce:

In a workspace with multiple projects search any string
Results are being returned from .git folder and .gitignore file
Search.exclude is set in workspace/User/and individual projects like so:

This only happens when I am searching while using a workspace. If I open any of the projects on their own I do not have this issue.
Does this issue occur when all extensions are disabled?: Yes

Please check these tips https://github.com/microsoft/vscode/wiki/Search-Issues#search-including-too-many-results


Is it possible to use different styles on the same page?

it is not an issue, I did not find where to ask this question (on stackoverflow?)
this question has been asked before https://github.com/highlightjs/highlight.js/issues/862
but i didn't understand the answer, i was missing a concrete explanation of the answer
In the example below, I put two link tags in the body
Of course it doesn't work, I imagine there is a method.
<!doctype html>

    <link rel="stylesheet" href="highlight/styles/codepen-embed.min.css">

    <link rel="stylesheet" href="highlight/styles/codepen-embed.min.css">

        //javascript
        let x=5;

    <link rel="stylesheet" href="highlight/styles/arta.min.css">

        //GLSL
        uniform1i ui;
        void main() {
           gl_FragColor = vec4(1);
        }

Not trivially, but with CSS of course this wold be possible just with simple application of another class.  You'd have to modify your themes accordingly to allow this sort of thing.  That might be trivial though if you are using a pre-processor such as SCSS, etc...
/* theme one */
.hljs.theme_one .hljs-variable {
  color:blue
}

/* theme two */
.hljs.theme_two .hljs-variable {
  color:red;
}

Not trivially, but with CSS of course this would be possible just with simple application of another class on each code block. You'd have to modify your themes accordingly to allow this sort of thing. That might be trivial though if you are using a pre-processor such as SCSS, etc...
/* theme one */
.hljs.theme_one .hljs-variable {
  color:blue
}

/* theme two */
.hljs.theme_two .hljs-variable {
  color:red;
}

I don't understand your answer. In the example I give, I would like to give 2 different themes in the same page. What should be changed in this example?

BUT you've also have to entirely rewrite the themes CSS to have every rule nested inside a theme class also:
.theme_two .hljs-tag .hljs-attr {
  color: #444;
}

Easy, but not trivial.

Closing as asked and answered.  If you have any further questions feel free to still respond.


Allows updates to null

Super quick PR to close #33. I know I never gave the author a chance to respond to #33 so if there's a specific reason why this shouldn't be implemented, feel free to close.
Thank you very much for the awesome lib!

RE: #29

Good fix thanks

The null values are now saved as NULL in the database, so that we can do queries with params like this: where: "parent IS NULL"


Mm 46549 gather intent cloud monthly subscription

Summary
Allow the customers to submit their feedback for new payment options while they are buying their monthly subscription
Ticket Link

https://mattermost.atlassian.net/browse/MM-46549
Related Pull Requests

Has server changes

Screenshots

Stripe metadata

Release Note
Allow customers to submit their feedback for wire transfers to buy monthly subscription.

E2E tests not automatically triggered, because PR has no approval yet. Please ask a developer to review and then try again to attach the QA label.

95% of this PR are the same changes that from Portal.

/e2e-test

Successfully triggered e2e testing!
https://git.internal.mattermost.com/qa/cypress-ui-automation/-/pipelines/251034

@Afsoon add a e2e test with lint fixes here https://github.com/mattermost/mattermost-webapp/pull/11371

Added the suggestion for CSS also updated the CSS and changes made in the other PR

gentle ping @AGMETEOR

@yasserfaraazkhan is this QA Review approved? I noticed you changed the tag but there is no approval or comments so double checking here

@yasserfaraazkhan is this QA Review approved? I noticed you changed the tag but there is no approval or comments so double checking here

This change is verified @Afsoon @marianunez. Added a e2e and also verified the manual scenarios mentioned in ticket


tf-idf implementation

A simple tf-idf implementation. Tests are included, as well as documentation. Modified README.md to no longer list tf-idf as near-sighted goal.

Awesome, thanks!


[OmniSci] Make codegen in OmniSci more friendly for Series processing

Currently, rows processing loop has (supposedly) unnecessary memory accesses. This causes a significant overhead when row size is quite small (e.g. binary operation for Series). We should investigate why all those memory accesses exist and try to remove them (completely remove or move out of the loop).

Let's track this here - https://github.com/intel-ai/omniscidb/issues/176


Use camelCase and remove "list" prefix from methods

Description of change
Use camelCase and remove "list" prefix from methods
Links to any relevant issues
Fixes https://github.com/iotaledger/wallet.rs/issues/1452
Type of change

Breaking change (fix or feature that would cause existing functionality to not work as expected)

How the change has been tested
Running examples
Change checklist

[ ] I have followed the contribution guidelines for this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have checked that new and existing unit tests pass locally with my changes

The secrete manager changes require the latest commit from iota.rs, so https://github.com/iotaledger/wallet.rs/pull/1460 should be merged before this PR


re-added STK files...

I initially was going to do some fancy git footwork to remove the binary files in my PR, but I found
it was just easier to blow up my repo, re-fork, and re-add the files.

Okay, still kinda hoping to clear out the really weird stuff like the tcp server included in STK, but at least those are just text files.


Adding links shouldn't ask to upload images, should use site preview

Hi, great project.
Adding links shouldn't ask to upload images, should use site preview.
Thanks

thanks, just implemented in this commit: https://github.com/sepehr-safari/pinstr/commit/fd228a8af8e1ccbd57dcd8ea519fbdf414d27db1

Just to know, this was not applied on pinstr.app site yet, right?


Feat/cs ds equality

This PR fixes the issue described in #370
The problem was in the context-box since it did not update the Desired State after builder finished processing. This lead to the difference between these two.
This PR also improves the printConfig() function, with a small test to check if the prints are equal for both states.

@Despire could you please check my latest commits before we merge it?

why is the check if dbConf.DesiredState != nil {  needed ? shouldn't the config from builder also have desired state set to nil ?

Thanks for feedback @Despire :)

why is the check if dbConf.DesiredState != nil { needed ? shouldn't the config from builder also have desired state set to nil ?

Image a case, where Builder will start building the infra, and while Builder is working, a request for deletion comes.
The deletion request will set DesiredState to nil and would wait until the Builder finishes.
When the Builder is finished we need to save the current state as well as desired state, since Builder updates both of them. So if we would not check, if the config in a database does not have a DesiredState set to nil, the SaveConfigBuilder would overwrite it with the desired state from before the deletion request, resulting in a DesiredState not being nil, even though it should have been because of the deletion request.

thx for the explanation @MiroslavRepka, didn't considered the deletion at all :),
the changes looks good 👍


Add  SonarCloud badges to README.MD

Add SonarCloud badges to README.MD file

Will be set by #111


Huge Size

Describe the solution you'd like
Does this have to be 500mb? Can you add a minimal install to the pipeline?
Thanks!

It is huge, I know.  If you're not interested in the examples, you can unselect that folder when importing.  If you're not interested in the default races, you can drop out the Content folder. If you want just the default races, you can drop everything in Content except for the core folder.  We don't have an automated release, so each one (currently) is me basically doing it all by hand.
I'll leave this out here for discussion on how best to implement some sort of automated process, or manual breakup of releases in the future.

brililant thanks!

So i included just the default races as above but i am getting errors about missing default animation controller
Where would I find that in the package archive?

Thanks :-)

Assets/UMA/Content/Example/Animators/Locomotion.controller
You might want to join the discord, there's a lot of people on there:
https://discord.gg/6f6Unr9G

Closing. Will add work item to break out sample data.


Handle some cases where two PDFs were concatenated together.

Hi!
We've had a report with a PDF file which PDF.js could not open, but other PDF readers (e.g. Preview, Adobe Reader), could open without issues.
In investigating this, it turns out the PDF file is actually two different PDFs which were concatenated one after the other into a single file. Obviously not a valid PDF, but given that other PDF readers will show one of the PDFs in this case, we probably should too (sigh).
There was some previous discussion of this on issue #5255.
I've found the issue preventing us from showing one of the PDFs - it's because in the XRef indexObjects method in obj.js, we don't stop reading when we hit a %%EOF marker, and we will happily overwrite things in the entries array.
Stopping at %%EOF has the side effect of breaking reading linearized PDFs that have a broken startxref value (because they contain one at the beginning too), so as an alternative fix I have implemented a check where we don't write into the entries array if a value is already present. I'll be pushing up a PR with that shortly.
I've uploaded the file here: https://www.dropbox.com/s/pvcuea8fmg1nc3c/21063565.pdf?dl=0
Thanks
Jordan

Fixed by #5910.


replace mathjax with katex

In the new editor we'll be using KaTeX  for rendering math formulas, so it makes sense to get rid of mathjax altogether. Also we should check if the server side rendering to HTML by KaTeX is an option.

\underbrace and \overbrace are still not supported, but used in serlo. So I'll close this for now...

Ok, so as the community constantly adds support for new functions we might think about switching to katex and using mathjax as fallback. However this also increases the load, as both katex and mathjax have to be loaded.
Also currently KaTeX doesn't break automatically if the parents width is too small. This has to be adressed too, so we need a workaround or wait for KaTeX to add the feature: https://github.com/Khan/KaTeX/issues/327

Please check out https://github.com/mathjax/MathJax-node


ESLint should be a devDependency

Eslint as a regular dependency causes some build failures, it should be a dev dependency

Thanks @JeremyRubin
Will be fixed in the next version.


Control parameter 'datetype'='init' is ignored

I am testing the rdams-client Python utility to submit subset requests on ds084.1 but I'm experiencing problems in getting the date range to include data based on model initialization date/time instead of valid date/time.
Here is the control info I'm submitting:
control = {
         'dataset' : 'ds084.1',
         'date':'202003250000/to/202003250000',
         'datetype':'init',
         'param':'TMP/R H/ABS V',
         'level':'ISBL:850/700/500',
         'nlat':60,
         'slat':30,
         'elon':30,
         'wlon':-1,
         'product':'3-hour Forecast/6-hour Forecast/9-hour Forecast/12-hour Forecast'
         }

Here is the list of files I'm getting:
gfs.0p25.2020032412.f012.grib2
gfs.0p25.2020032418.f006.grib2

Here is the list of files I'm expecting with the 'datetype'='init' parameter activated:
gfs.0p25.2020032500.f003.grib2
gfs.0p25.2020032500.f006.grib2
gfs.0p25.2020032500.f009.grib2
gfs.0p25.2020032500.f012.grib2

So it looks like the control parameter 'datetype'='init' is ignored.
Manually submitting the following request from the GUI on the ucar webpage, which carries the exact same information as the control file,  I get the expected list of files.
Subset details:

- Start date: 2020-03-25 00:00
- End date: 2020-03-25 00:00
- *Initialization (reference) dates
- Parameter(s):
    Absolute vorticity
    Relative humidity
    Temperature
- Vertical level(s):
    Isobaric surface: 500 mbar
    Isobaric surface: 700 mbar
    Isobaric surface: 850 mbar
- Product(s):
    3-hour Forecast
    6-hour Forecast
    9-hour Forecast
    12-hour Forecast
- Spatial subsetting (bounding box):
    Latitudes (top/bottom): 60 / 30
    Longitudes (left/right): -1 / 30

Hi,
Thank you very much for pointing this out. The code was definitely skipping the datetype option, but I believe this has been fixed. Let me know if you're still experiencing problems, otherwise I'll close the issue in a week or so.

Thank you! By now I can't see any new commit in the repo. Can you please check if you've pushed them? :)

The changes were made in another repo. This repositories scope is limited to client-side use cases.

Ok! I just tried it out and it's working perfectly now! Thank you very much!


Avoid autocompletion after semicolon and paren

Closes #660

My full approval to anything you want to do in this project ... unless you explicitly ask me to review :rose:

Great, thanks :smile: I still like the occasional review or pointer.


Using OpImageSampleProjDrefExplicitLod emits an exception with Lod != 0

When the op OpImageSampleProjDrefExplicitLod is used on a sampler2DShadow, this error is emitted:

SPIRV-Cross threw an exception: textureLod on sampler2DArrayShadow is not constant 0.0. This cannot be expressed in GLSL.

However, despite the sampler type showing as incorrect in the error (I am using a sampler2DShadow and not a sampler2DArrayShadow), this works fine in GLSL so it can be expressed despite the error saying other wise.
It does not appear to be against the specification to do so either:
https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/textureLod.xhtml
Cheers and any help is appreciated 😄

But textureLod() doesn't exist for sampler2DArrayShadow or samplerCubeShadow. The check here is supposed to catch those two cases, but it appears to be triggering incorrectly. What SPIR-V are you running this on?

Pretty sure this is an oversight. I'll check it out, won't need a repro case.

My use case was my own d3d9 to vk layer d9vk with pretty much any game that uses depth compare (ie. A Hat in Time etc.) I'll try and fish out a broken shader tomorrow.

Ping @Joshua-Ashton. Did you get a repro case?

Closing as there's been no repro.


Cannot open UBUNTU or any instance with public ip address

After creating an instance, i get the public ip address. But unable to open that page. It says Page not found.
Even in git bash. It says connection timed out when entering the public IP address.
please help me solve this issue as soon as possible.

have u opend port 22 or choose pemfile path and u connect with that machine


make download_tools command is missing in the main README.md

It would be nice if we had the make download_tools command in the main README.md. I was following the guide and got stuck for a moment due to missing commands like kubectl or htpasswd.

The issue was resolved by https://github.com/openstack-k8s-operators/install_yamls/commit/534095dbfb64968223ef21e6c79a809fd6c93cab from @fao89.


Move performance tests and benchmarks from "make test" to a new "make benchmark"

Original issue: https://charm.cs.illinois.edu/redmine/issues/1872

A non-trivial portion of "make test" jobs is spent on performance testing. These tests should be moved to a separate "make benchmark" so that automated unit and integration testing can complete more quickly.

Original date: 2018-04-18 16:21:23

And at least some of the results of "make benchmark" should be plotted online somewhere every night based on some of the autobuilds whose performance we care about tracking. It would be useful for diagnosing performance regressions. We already do this for netlrts charm++ pingpong on the PPL wiki page.
I would want example/ampi/pingpong and tests/charm++/pingpong at a minimum. Potentially some reduction/broadcast tests, Converse/LRTS tests, and maybe some example programs to all have that.
This could be a good Charmworks intern task.

Original date: 2018-08-10 19:05:40

~~https://charm.cs.illinois.edu/gerrit/4218~~ https://github.com/UIUC-PPL/charm/commit/a01ac23867bb818ba6ac01da65f088558c54feb6

Original date: 2019-04-02 18:49:16

We should merge the 6.10-targeted Zero Copy patches, and as many other patches under review that add new tests/examples as possible, before I proceed with this reorganization.


Deprecate classification from the API

Description of change
The frontend should use the new onelist_group_tier field instead which takes into account a tier inherited from the Global Headquarters as well.
Checklist

[x] Has a new newsfragment been created? Check changelog/README.rst for instructions
[ ] Have any relevant search models been updated?
[ ] Have any relevant fixtures (fixtures/test_data.yaml) been updated?
[ ] Have any relevant select-/prefetch-related field lists in the views and search apps been updated?
[ ] Has the admin site been updated (for new models, fields etc.)?
[ ] Has the README been updated (if needed)?

Depends on #1238


Can't ignore unmaintained crates?

Describe the bug
I'm trying to ignore the fact that a crate is unmaintained, but I don't know how.
Is the ignore field only for yanked crates? If so, how do I ignore unmaintained crates?
To reproduce
This is what I tried:
[advisories]
version = 2
ignore = [
  "safemem",
<EMAIL_ADDRESS>]

Resulting in:
error[unmaintained]: safemem is unmaintained
    ┌─ Cargo.lock:497:1
    │
497 │ safemem 0.3.3 registry+https://github.com/rust-lang/crates.io-index
    │ ------------------------------------------------------------------- unmaintained advisory detected
    │
    = ID: RUSTSEC-2023-0081
    = Advisory: https://rustsec.org/advisories/RUSTSEC-2023-0081
    = The latest crates.io release was in 2019. The repository has been archived by the author.

…

    = Announcement: https://github.com/abonander/safemem
    = Solution: No safe upgrade is available!
    = safemem v0.3.3
      └── base64 v0.9.3
          └── wasm-bindgen-cli-support v0.2.90
              ├── cargo-run-wasm v0.3.2
              │   └── run_wasm v0.16.0-alpha.1+dev
              └── re_dev_tools v0.16.0-alpha.1+dev

warning[yanked-not-detected]: yanked crate was not encountered
   ┌─ /Users/emilk/code/rerun/rerun/deny.toml:34:4
   │
34 │<EMAIL_ADDRESS>   │    ^^^^^^^^^^^^^ yanked crate not detected

warning[yanked-not-detected]: yanked crate was not encountered
   ┌─ /Users/emilk/code/rerun/rerun/deny.toml:33:4
   │
33 │   "safemem",
   │    ^^^^^^^ yanked crate not detected

advisories FAILED, bans ok, licenses ok, sources ok

cargo-deny version
cargo-deny 0.14.21
What OS were you running cargo-deny on?
MacOS
Additional context
No response

The ignore field is for yanked crates or advisories, in the case of unmaintained crates you use the advisory for it, but I can see how using the crate spec would also make sense.

Ah that makes sense - thanks. Ignoring "RUSTSEC-2023-0081" works as expected.


Flag machine specific configurations like paths as machine scoped

Machine Scope for configurations were introduced in 1.34 which will help in remote set ups.
I would like to encourage extension authors to adopt to machine scope if you have any settings that you think they are scoped to machine like eslint.nodePath
For example, If you have settings that allow users to customize an executable path that exists on the machine in which extension is running, then you can tag such settings as machine scoped. This will let VS Code to read this setting from the location where the extension is running.

When extension is running locally, VS Code reads this setting only from User settings on the same machine.
When extension is running remotely, VS Code reads this setting only from Remote settings on the remote machine.

If the extension is running remotely in a remote set up, then this will help the extension not to read paths configured in local machine user settings and bail as the paths not exist in the remote machine.
Note that machine scoped settings are overridable only in user or remote settings.

As dicussed I made the eslint.runtime as machine setting. For the nodePath we need an extended solution in the settings space.

Made "eslint.nodePath" machine overridable.


Custom HTML rendering

Is there a way to customize how the markdown is rendered to HTML? Especially for publishing.
I want to use StackEdit to write posts for my WordPress blog. I use the SyntaxHighlighter Evolved WP plugin to render code blocks. To use this plugin, I just paste the raw code between [sourcecode][/sourcecode] tags, without any prior formatting. But when StackEdit publishes my post to Wordpress, it does its own HTML formatting, so I can no longer use the [sourcecode] tag.
Could it be done with an extension? Is there some kind of callback like function onRenderingCodeBlock(code, language)?

You can do that with templates and custom handlebars helpers in the v5.


Add entry points that take an "Error"

In cases where we have an underlying error available,we should be able to pass it straight through rather than needing to generate a string ourselves from it (which has issues with localization, and is just more work).
We should definitely add this for the error level, and maybe consider it for the warn and fatal levels as well. Those wouldn't be as useful under Crashlytics, because we wouldn't want to report Crashlytics non-fatals for warnings or fatals, but assuming we do eventually move to Sentry, it has enough better categorization that it might be useful to send errors with a different tag on other levels for those that can be organized separately (for example, network errors could be sent at the warn level, giving the ability to track the frequency, how we do sometimes now by making network errors an analytic event, of those while still not having them generate issues automatically like errors do.
One design element that needs to be resolved before doing this: do we want to accept ALL errors, or do we want to force errors to opt in in some way, so that we can try and enforce the same redaction rules as we do for passing in objects.
On android the class involved isn't Error (java.lang.Throwable maybe?), but we shoudl do the same thing.

This was completed in #105.


Passcode

The user upgraded from 2.0.90 (509) live to the latest version but was not able to log in even after using the same passcode.

Duplicate of #6297

fixed


[DO NOT MERGE] Make data-ga4-attachment-link appear on all attachment links

Do not merge as we are deploying/changing the attachment tracking in a stepped process, so that we don't affect the existing file_download tracking .
Previously we were tracking HTML attachments as 'attachment' and all other attachment links as file_download using JS. This change makes it so that every attachment link will get the type 'attachment' set on it in our ga4-link-tracker.js in publishing components.
https://trello.com/c/MHFXw6H2/552-change-filedownloads-on-attachment-pages-to-have-attachment-instead-of-generic-download-type
⚠ This repo is Continuously Deployed: make sure you follow the guidance ⚠
Follow these steps if you are doing a Rails upgrade.

I'm closing this pull request as Whitehall no longer renders attachments itself following #7932.


Facet filter for parent id in a subcollection index

Hey! I'm wondering if there's a way to create a facet filter for the parent id of documents in a subcollection.
So if the Algolia Index is synced from a firestore subcollection like parent/{parentId}/child/{childId} it would be great to be able to see all documents from a certain parentId
Thanks!
David

Sure, let me think about it.

Would you want the parent Id or the parent name?


does not build/run on OSX (M1 => darwin-arm64)

I tried to get the toolkit building and running on my M1 MacBookPro without success.
When I clone the repo and run 'yarn install' I get an error message:
HTTPError: Response code 404 (Not Found) for https://github.com/electron/electron/releases/download/v9.4.4/electron-v9.4.4-darwin-arm64.zip
When I change the Version for electron in packages.json to "electron": "^13.0.0" yarn install runs fine.
When I then run yarn run build, it build also fine except that the signing fails. yarn run dev fails.
When I run the built toolkit I only get a white/blank/empty window.
Any ideas how to get this running on Apple-silicon?

Unfortunately, I don't have any leads on this (not an M1 Mac user). I think we've got someone on our team here at Indicio that might be able to spend a bit of time looking into this but I wouldn't count on this being fixed in the near future, unfortunately.

If it is an option for you to upgrade to a newer version of electron (one that supports M1) and you can confirm that it works fine (on 'normal' systems), I could check it out on M1 and if there are Problems, try to find out why.

This is working fine on my M1 Mac.

Thanks for the info @froissant; I suspect this was fixed with our recent upgrade to Electron 12. Closing as fixed!


Automated Package Upgrade

:tada: This PR is included in version 5.3.62 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add dojo name to My events's tickets

When visiting the user's events from /dashboard/tickets (user menu->my events), the Dojo name is not displayed. It's fine if you're part of a single Dojo, but a bit disturbing when running multiple. I've come to ask myself "from which dojo is this event ?" as part of the process of asking myself if I'm around at that time.
I would recommend adding the Dojo name under the event name.
Thoughts @conoramurphy  ?

Ya makes sense to me.

has this been fixed yet?  If not i'd love to work on it!  Thanks


client connection with multiple GET requests

Hi
With this framework is it possible to connect to a REST SSL endpoint and do multiple GET requests without disconnecting, for example (using the example code as and
this is an excerpt from the SSL example...
http::ssl::client my_https_client{ctx};
const auto & host = "www.google.com";
uint32_t port = 443;

const auto & on_connect = [](auto & session){
    http::base::out("Successful connected!");

    session.do_handshake();
};

const auto & on_handshake = [&host](auto & session){
    http::base::out("Successful handshake!");

    boost::beast::http::request<boost::beast::http::string_body> req;
    req.version(11); // HTTP 1.1
    req.method(boost::beast::http::verb::get); // GET
    req.target("/ENDPOINT1");
    req.set(boost::beast::http::field::host, host);
    req.set(boost::beast::http::field::user_agent, BOOST_BEAST_VERSION_STRING);
    session.do_write(std::move(req));
};

const auto & on_receive = [](auto & res, auto & session){
    cout << res << endl;
    // session.do_close();
    http::base::processor::get().stop();
};

my_https_client.invoke(host, port, on_connect, on_handshake, on_receive);

uint32_t pool_size = boost::thread::hardware_concurrency();
http::base::processor::get().start(pool_size == 0 ? 4 : pool_size << 1);
http::base::processor::get().wait();

return 0;

}
If i make the changes above (not close the session in the on_receive lambda) is there a way i can
do something like send a request to a different endpoint (on the same host)  without closing the connection?

I leave it here as a solution...
auto make_request(http::method_t method,
                  boost::beast::string_view host,
                  boost::beast::string_view target){
    boost::beast::http::request<boost::beast::http::string_body> req;
    req.version(11); // HTTP 1.1
    req.method(method); // GET
    req.target(target);
    req.set(boost::beast::http::field::host, host);
    req.set(boost::beast::http::field::user_agent, BOOST_BEAST_VERSION_STRING);
    return req;
}

int main(){
...
    const auto & on_handshake = [&host](auto & session){
        http::base::out("Successful handshake!");
        session.do_write(make_request(boost::beast::http::verb::get, host, "/ENDPOINT1"));
    };

    int ep = 1;

    const auto & on_receive = [&host, &ep](auto & res, auto & session){
        cout << res << endl;

        if(ep == 1){
            ep = 2;
            session.do_write(make_request(boost::beast::http::verb::get, host, "/ENDPOINT2"));
            return;
        }

        session.do_close();
        http::base::processor::get().stop();
    };

    my_https_client.invoke(host, port, on_connect, on_handshake, on_receive);
...
}

Perfect, this is what i was generally looking for!
One final question, it looks like i can send on the new endpoint "/ENDPOINT2" only if i receive a packet. how would I send to ENDPOINT2 without already in the on_receive handler ?
Many thanks for the prompt reply!

HTTP 1.x is not supported multiple requests over a single connection!
http::ssl::client conn1{ctx};
http::ssl::client conn2{ctx};
// as above

Something like this! :)

thank you for the info!


RATIS-957. Update .asf.yaml to define jira notifications strategy

What changes were proposed in this pull request?
Although .asf.yaml was recently updated with notifications settings (#116), pull-request-available label and PR link are not being added to RATIS Jira issues.  This PR attempts to fix that.
Config was taken from repos where this works correctly (eg. Hive), as well as from Ratis current settings (although link label seems to be added automatically to that).
https://issues.apache.org/jira/browse/RATIS-957
How was this patch tested?
Unfortunately there is no easy way to validate before merge.

Thanks @adoroszlai for working on this. I have committed this.

Thanks for @bshashikant reviewing and committing it.  Now that it's merged, verified in RATIS-953: link and worklog were added, but no label (yet?).


Support SOP in Entity Framework

I am interested in integrating/supporting EF in this pure managed code, NoSQL engine code library (see https://sop.codeplex.com/ for details). I can see this to be a win-win feature for everybody. SOP provides solution to a lot of data persistence problems from personal repo/small shop to the Enterprise.
In addition, EF itself may benefit from tremendous, high speed virtualized RAM caching SOP provides.

Hi Rowan/et al,
Pls. let me know if you need anything from me.
I am not too much familiar w "engagement protocol" on things like this. But I am open & flexible. I can provide anything within my limits to get some traction on this. :)
Pls. feel free to provide guidance/direction or any feedback when ready.
Thx much, sincerely,
-G.

Hey,
Thanks for reaching out to us. At the moment our code base is still very much in-flux so there isn't really any documentation etc. as things change day-to-day. The best way to get started is to take a look at the code for the providers we have and then ask questions as you run into issues. The providers in our code base are mostly relational at the moment, but there is an InMemory provider that may be helpful to you.
The https://github.com/npgsql/npgsql project is also working on EF7 support at the moment, so you may find some useful discussion between the EF and npgsql teams in our issue trackers (that is a relational provider though, so their implementation will be quite different to yours).
Of course, as our code base stabilizes and we get closer to an RTM we will provide documentation etc. for provider writers.
~Rowan

Thx, I'll look into those (sample) provs & d InMemory version.

SOP Synopsis
SOP is a modern, scalable, portable, lightweight & highly adaptive object management system packaged in a code library. What separates SOP from the rest of the pack is its proven adaptability and being written in managed code (c#), it can be adapted quite quickly and easily across diverse use-cases, e.g. - from very thin use as a RAM virtualizer/high performance persisted data cache or on the opposite end, used as a data mgmt platform for providing high performance, enterprise scalable/durable data storage solution that runs on a new env't like the Azure cloud to take advantage of massive distributed processing & geo-redundancy it provides. All these while achieving its primary goal of reducing (and thus, optimizing!) persistence layers involved in application data mgmt to bare minimum.

UPDATE: the just released version of SOP B-Tree .Net (v4.8) now sports LINQ to Objects extensions which fully enables complete LINQ to Objects functionality inter-operability. Somehow, LINQ to Entities support suddenly becomes less of a priority for this code library.
I wonder though whether there is still a need for this feature? 'other than perhaps, for using LINQ to Entities as an abstraction "layer", which anyway may not be necessary for SOP due to it operates in the context of the user code/objects (POCOs), thus, it already puts the user code as 1st class citizens.
LINQ to Objects seems to be ideal, 'talking about "query language" per se. Pls. advice on your opinion, if/when time permits. 'just wanted to get your point of view.

Docs on writing a provider at https://docs.microsoft.com/en-us/ef/core/providers/writing-a-provider
This isn't a provider we would put in the core project, it should be a community maintained provider.