Incorrect use of torchaudio's rnnt_loss

It is suprising that no one has noticed that the current code is feeding the output of
log_softmax to torchaudio's rnnt_loss, which is not correct.
torchaudio's rnnt_loss accepts only logits as input.
I am wondering whether anyone has succeeded in training a transducer model
with the current usage of torchaudio's rnnt_loss using speechbrain.

The relevant code is listed below:
https://github.com/speechbrain/speechbrain/blob/24720c1532355dfeeda78304939dcc80283244bf/recipes/CommonVoice/ASR/transducer/train.py#L75
https://github.com/speechbrain/speechbrain/blob/24720c1532355dfeeda78304939dcc80283244bf/recipes/CommonVoice/ASR/transducer/train.py#L136-L138
https://github.com/speechbrain/speechbrain/blob/24720c1532355dfeeda78304939dcc80283244bf/recipes/CommonVoice/ASR/transducer/hparams/train_fr.yaml#L183-L185
https://github.com/speechbrain/speechbrain/blob/24720c1532355dfeeda78304939dcc80283244bf/speechbrain/nnet/losses.py#L59-L77

I'm trying to train an RNN-T model... Have you tried to train a model with this error fixed? How much was the difference?

No, I have not used it to train a model. I just spotted this bug while looking at the code.

Hello @csukuangfj , thank you for reporting the issue,
let me do a fix

It's strange but I didn't notice any improvements during the training process, after fixing this bug it produces even worse results during the validation.. I'll make the update once I decode it on the test set

It's strange but I didn't notice any improvements during the training process, after fixing this bug it produces even worse results during the validation.. I'll make the update once I decode it on the test set

softmax outputs - Epoch loaded: 30 - test loss: 31.53, test WER: 55.05, test CER: 41.22
(FIXED) logits - Epoch loaded: 30 - test loss: 30.48, test WER: 54.91, test CER: 41.20`

Hello,
This issue has been solved with #1368, so I'm closing this issue.
Thanks again for reporting this problem! :-)


JBPM-4833 - Data Modeller MVP, Tests, and related improvments

-DataObjectBrowser, MPV improvements + tests + i18n review

@pefernan
Hello Pere, would you mind review this PR?
Thanks

+1

No idea how this could be added, btw, fixed.

6.4.x: 78a8e665131c3fc31a360ac3085561bba1832560
@wmedvede please don't forget to send this to master ;).


Cut-off long abstracts at full word

Original report on BitBucket by Taina Jääskeläinen.

We have a character limit point after which long abstracts are cut off in detailed study view.
It would be better if the cut-off was done at full word, if possible. Three dots still needed.
Related to resolved #414.

This has actually already been done at some point.
Abstract is truncated by truncate from lodash with the separator option: [options.separator] (RegExp|string): The separator pattern to truncate to.
import { truncate, upperFirst } from 'lodash';
...
function truncateAbstract(string: string): string {
  const trimmedString = string.trim();
  return truncate(trimmedString, { length: 500, separator: ' ' } );
}


[ZEPPELIN-1924] Fix "this._isNumeric is not a function" error

What is this PR for?
After #1815 was merged,
uncaught TypeError: this._isNumeric is not a function
    at ColumnSettings._numericValidator [as validator] (handsonHelper.js:172)
    at handsontable.js:5181

is shown when click "Numeric" in the result table like below.

Since ES6 no longer supports autobind for this, seems it needs to be bound in the constructor.
What type of PR is it?
Bug Fix
What is the Jira issue?
ZEPPELIN-1924
How should this be tested?
To reproduce
In master, go to Spark tutorial note and click

Numeric. Then the error msg will be shown in browser dev console.
With this patch, this error msg won't be shown up anymore :)
Questions:

Does the licenses files need update? no
Is there breaking changes for older versions? no
Does this needs documentation? no

Tested, LGTM


Update install_bindcraft.sh

Replaced loose conda commands with $pkg_manager

These have been left on purpose for now as they are more consistent in mamba environments. Mamba really only makes a difference during installation

I see. My installation failed until I made the change.


Separate skdb/wasm-b64 exports entrypoint

Usage:
import { SKDB, fetchWasmSource } from 'skdb'

const skdb = await SKDB.create('db-name', fetchWasmSource)

My guess is that you just don't import the wasmBase64 now and rely on the fallback behaviour?

My guess is that you just don't import the wasmBase64 now and rely on the fallback behaviour?

Yes, the wasmBase64 behaviour would need to be used explicitly as seen in my initial comment.

Looks good. I need to transplant this to an upstream repo otherwise it will get clobbered. Will do that now. So I'm closing this and you'll see the changes get pushed through with the next release.


Feature: Improve ReductClient initialization

Current problem
Client initialization is verbose and counter-intuitive:
        ServerProperties properties = new ServerProperties(false, "<IP_ADDRESS>", 8383);
        HttpClient httpClient = HttpClient.newHttpClient();
        ReductClient client = new ReductClient(properties, httpClient, "my-token");

Possible solution
Let's pass the URL and token to ServerProperties and use the builder pattern because in the future we will need timeout and other options:
        ServerProperties properties = ServerProperties
            .url(:http://<IP_ADDRESS>:8383)
            .apiToken("my-token");

        HttpClient httpClient = HttpClient.newHttpClient();
        ReductClient client = new ReductClient(properties, httpClient);

Additionally, lets add a constructor with a default HTTP client.
Additional context
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

I've started to work on the issue


linkedin account was banned after I use this scrapper

after I got several result , the account was banned from linkedin , so how to avoid this happen if I register another account ?

and I referenced your reply on scrapedin , there was an waitingtime parameter for that .
but for scraping multiple profiles , I use this scrapedin-linkedin-crawler , I did not find the same parameter in the src js files yet . could someone give some hint to deroute this issue? thanks

I just saw the profile wait time parameter in conf.json file , I will try enlarge that one

Hi @tszjqgs

after I got several result , the account was banned from linkedin , so how to avoid this happen if I register another account ?

Honestly, I don't know how, because it's something on LinkedIn that detects the account usage and decides to block, and I'm unaware of that logic.
I suppose that's something like watching hundreds of profiles, or due to very fast clicking, so you could try:

increase wait time seconds, to decrease the number of viewed profiles per seconds
remove that line on /node_modules/scrapedin/profile.js so there will be no clicking on the page (however you'll collect less information)
you can also remove that line as well that scrolls on the page, maybe they're listening to the scroll event as well
I'm also not sure but logging via cookies maybe helps too.
Maybe requiring them to unblock your account (proving with an Id or something they request) you could have an "unlimited" account, I never did that before, but you can also try.

Please try these options and give us some feedback.

thanks , enlarge that parameter helped

thanks, i submit the id and see what can be done after that ,still waiting response from linkedin
发件人：Wagner Leonardi<EMAIL_ADDRESS>发送时间：2019年10月17日(星期四) 22:50
收件人：linkedtales/scrapedin-linkedin-crawler<EMAIL_ADDRESS>抄　送：tszjqgs<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>主　题：Re: [linkedtales/scrapedin-linkedin-crawler] linkedin account was banned after I use this scrapper (#17)
Hi @tszjqgs
after I got several result , the account was banned from linkedin , so how to avoid this happen if I register another account ?
Honestly, I don't know how, because it's something on LinkedIn that detects the account usage and decides to block, and I'm unaware of that logic.
I suppose that's something like watching hundreds of profiles, or due to very fast clicking, so you could try:
increase wait time seconds, to decrease the number of viewed profiles per seconds
remove that line on /node_modules/scrapedin/profile.js so there will be no clicking on the page (however you'll collect less information)
you can also remove that line as well that scrolls on the page, maybe they're listening to the scroll event as well
I'm also not sure but logging via cookies maybe helps too.
Maybe requiring them to unblock your account (proving with an Id or something they request) you could have an "unlimited" account, I never did that before, but you can also try.
Please try these options and give us some feedback.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

and where to inject the cookie in the scrapedin-linkedin-crawler ？ in which file ? there is not an comparable place in scrapedin (which for single profile scraping)

I updated now the project to you place your cookies file in config.json see changes here: and where to inject the cookie in the scrapedin-linkedin-crawler ？ in which file ? there is not an comparable place in scrapedin (which for single profile scraping)
you can either git pull or apply those changes manually in your code

thanks, I redid the git clone over again and will try it out
发件人：Wagner Leonardi<EMAIL_ADDRESS>发送时间：2019年10月18日(星期五) 22:24
收件人：linkedtales/scrapedin-linkedin-crawler<EMAIL_ADDRESS>抄　送：tszjqgs<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>主　题：Re: [linkedtales/scrapedin-linkedin-crawler] linkedin account was banned after I use this scrapper (#17)
I updated now the project to you place your cookies file in config.json see changes here: and where to inject the cookie in the scrapedin-linkedin-crawler ？ in which file ? there is not an comparable place in scrapedin (which for single profile scraping)
you can either git pull or apply those changes manually in your code
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

hi wangner , I installed the Editthiscookie extension in browser , and loggedin the linkedin , than I click the cookie button , click the export , and directly copy the content into the config.json at the cookie area , but the npm start can not start welll, could you please provide a user guide about the coockie useage ? thanks
发件人：Wagner Leonardi<EMAIL_ADDRESS>发送时间：2019年10月18日(星期五) 22:24
收件人：linkedtales/scrapedin-linkedin-crawler<EMAIL_ADDRESS>抄　送：tszjqgs<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>主　题：Re: [linkedtales/scrapedin-linkedin-crawler] linkedin account was banned after I use this scrapper (#17)
I updated now the project to you place your cookies file in config.json see changes here: and where to inject the cookie in the scrapedin-linkedin-crawler ？ in which file ? there is not an comparable place in scrapedin (which for single profile scraping)
you can either git pull or apply those changes manually in your code
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

Hi @tszjqgs I'm closing this issue because there's a lot of questions on the same issue and I'm getting lost trying to answer all of them.
Please open another issue if you still have some problem, thank you.


na_tolerance feature

Adds to df_parse_table two predefined variables:

na_tolerance_r: int = 1,
na_tolerance_c: int = 1
This variables give the user the option to ignore a certain number of N/A cells when parsing for tables from corners.
Useful for cases in which tables might have merged cells or blank rows between the headers and the content.

With default value 1 it behaves as the original.

Let me know if you are good with these changes to your PR and we can merge it.  It probably would also be helpful to integrate these features into the CLI if you want to take a look at that.

Added nacount to the CLI and fixed a bug.  I'll bump the version and merge this Monday.


WMS: GM forces spell to surge

Wild Magic Surge automation could have an option to enable a request to the GM to roll the d100 table instead of rolling it right away automatically, just to give more drama to the thing.
A function to track Favored Enemy feature in an actor's sheet and the types of enemy (could be compatible with the Revised Ranger too), the bonus and damage would be applied automatically.
The upcoming automated cover feature could have an detection for the Sharpshooter feature, making the target not benefit from bonus AC for attacks coming from the actor with the feature.

Some initial thoughts on your 3 suggestions.

While I don't think the complexity of prompting the DM to roll on the table is worth the gain from tension, the PHB sorcerer does have the caveat of "at any time, the DM can cause a spell to surge" in the Tides of Chaos feature. With some clever coding and application by the user, we may be able to provide some of this functionality.
Note: I would not be opposed to it in general if other members of the team have interest in your suggestion as written.

Favored Enemy in the PHB has little mechanical automation for which 5e Helpers can assist and I do not intend on supporting UA features (in addition to wanting to leave the Revised Ranger firmly in the past).

Detecting Sharpshooter and not applying the attack penalty due to cover is a pretty good suggestion and we'll take that into consideration.

In summation:

Low priority, but not outside the scope of 5e Helpers
Will not address
Good idea, we will tackle this one for sure.

Thank you for your interest and support <3

And thank you for taking your time to read my suggestions <3
(Forgot to mention that the Spellsniper feature also ignores cover, same as Sharpshooter: half cover and three quarters cover)

Spellsniper is a good callout, thanks.

Feature (3) was implemented at some point in the 2.x series (lol).
Removing from current milestone and will re-evaluate Feature (1)

Adding this (feature 1) to the 4.1.0 (NEXT) milestone as it is focused on WMS automation.

Actually, closing this ticket as the remaining feature to be implemented is a duplicate of #43


Upgrade embedded Kotlin from 1.9.22 to 1.9.23

See https://github.com/JetBrains/kotlin/releases/tag/v1.9.23
Reviewing cheatsheet
Before merging the PR, comments starting with

❌ ❓must be fixed
🤔 💅 should be fixed
💭 may be fixed
🎉 celebrate happy things

@bot-gradle test this without PTS

I've triggered the following builds with parameters: -DenablePredictiveTestSelection=false for you. Click here to see all build failures.

PullRequestFeedback build

@bot-gradle test this without PTS

I've triggered the following builds with parameters: -DenablePredictiveTestSelection=false for you. Click here to see all build failures.

PullRequestFeedback build

The merge queue build has failed. Click here to see all failures.


Update logo

download and crop image from unsplash (https://unsplash.com/photos/3HYA5mEPRsE)
upload to hexmake (https://connect.thinkr.fr/hexmake/)
invert image in hexmake

Here's a draft:

Is there a way to crop the image keeping just the part inside the hexagon? I'm not seeing it...


RSpec 3 conversion: U-Z model specs

Converts U-Z model specs to the newer expect syntax

@miq-bot add_label rspec3, test

@Fryguy @jrafanie

Looks good @chrisarcand, can you extract the one change to a different PR?  https://github.com/chrisarcand/manageiq/commit/d6546f61a8847a3d6033f2ffef8f8fa423067464
I think that's right but we should have others review that part.

Sure can :+1:

Password change logic extracted to #5943

:tada:


🛠 [TASK] : Fix pallas library mithrill snapshot last chunk file issue

Summary
According to this documentation last chunk files are not immutable. So we need to skip this files and only read immutable data.
Description

Modify pallas library build_stack_of_chunk_names fn, exclude last chunk files because they are not immutable.
Update cardano-chain-follower crate with the fixed version.

related PRs:

https://github.com/txpipe/pallas/pull/426
https://github.com/txpipe/pallas/pull/454


Grunt Dependencies hardcoded (again)

Hellou
Again, this problem is related to https://github.com/AppGyver/grunt-steroids/issues/14. Peer dependency is fixed for grunt v0.4.4, while grunt v0.4.5 is the latest version.

This got worse since npm 3. Cause it got rid of peer dependencies.
First:
$ steroids connect

            __   AppGyver           .__    .___
     ______/  |_  ___________  ____ |__| __| _/______ 2
    /  ___\   ___/ __ \_  __ \/  _ \|  |/ __ |/  ___/
    \___ \ |  | \  ___/|  | \(  <_> |  / /_/ |\___ \
   /____  >|__|  \___  |__|   \____/|__\____ /____  >
        \/           \/                     \/    \/

>> Local Npm module "grunt-contrib-sass" not found. Is it installed?
>> Local Npm module "grunt-extend-config" not found. Is it installed?

Running "steroids-make-fresh" task
>> Local Npm module "grunt-extend-config" not found. Is it installed?
>> Local Npm module "grunt-contrib-clean" not found. Is it installed?
>> Local Npm module "grunt-extend-config" not found. Is it installed?
>> Local Npm module "grunt-contrib-coffee" not found. Is it installed?
>> Local Npm module "grunt-extend-config" not found. Is it installed?
>> Local Npm module "grunt-extend-config" not found. Is it installed?
>> Local Npm module "grunt-contrib-sass" not found. Is it installed?
Loading "steroids-module-compile-default-native-styles.coffee" tasks...ERROR
>> TypeError: grunt.extendConfig is not a function
>> Local Npm module "grunt-extend-config" not found. Is it installed?
>> Local Npm module "grunt-contrib-coffee" not found. Is it installed?
Loading "steroids-module-compile-coffeescript.coffee" tasks...ERROR
>> TypeError: grunt.extendConfig is not a function
>> Local Npm module "grunt-extend-config" not found. Is it installed?
>> Local Npm module "grunt-contrib-concat" not found. Is it installed?
Loading "steroids-module-concat-javascript.coffee" tasks...ERROR
>> TypeError: grunt.extendConfig is not a function
>> Local Npm module "grunt-extend-config" not found. Is it installed?
>> Local Npm module "grunt-contrib-copy" not found. Is it installed?
Loading "steroids-module-copy-javascript.coffee" tasks...ERROR
>> TypeError: grunt.extendConfig is not a function
Loading "steroids-module-compile-scripts.coffee" tasks...ERROR
>> TypeError: grunt.extendConfig is not a function
>> Local Npm module "grunt-extend-config" not found. Is it installed?
Loading "steroids-module-compile-views.coffee" tasks...ERROR
>> TypeError: grunt.extendConfig is not a function
>> Local Npm module "grunt-extend-config" not found. Is it installed?
Loading "steroids-module-copy-assets.coffee" tasks...ERROR
>> TypeError: grunt.extendConfig is not a function
>> Local Npm module "grunt-extend-config" not found. Is it installed?
>> Local Npm module "grunt-contrib-copy" not found. Is it installed?
Loading "steroids-module-copy-default-native-styles.coffee" tasks...ERROR
>> TypeError: grunt.extendConfig is not a function
>> Local Npm module "grunt-contrib-copy" not found. Is it installed?
>> Local Npm module "grunt-extend-config" not found. Is it installed?
>> Local Npm module "grunt-contrib-copy" not found. Is it installed?
>> Local Npm module "grunt-extend-config" not found. Is it installed?
>> Local Npm module "grunt-contrib-copy" not found. Is it installed?
>> Local Npm module "grunt-extend-config" not found. Is it installed?
>> Local Npm module "grunt-extend-config" not found. Is it installed?
Loading "steroids-setup-cloud-resources.coffee" tasks...ERROR
>> TypeError: grunt.extendConfig is not a function
>> Local Npm module "grunt-extend-config" not found. Is it installed?
Loading "steroids-setup-features.coffee" tasks...ERROR
>> TypeError: grunt.extendConfig is not a function

Running "steroids-check-project" task
Warning: grunt.extendConfig is not a function Use --force to continue.

After:
$ rm -rf node-modules && npm cache clean && npm install
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.

When looking at the package.json of grunt-steroids I really wonder why the dev dependencies are defined als peer dependencies:
https://github.com/AppGyver/grunt-steroids/blob/master/package.json#L48-L54
Did Grunt handle things that way in older versions? I’m confused.

I have the same issue with NPM version 3.3.12
npm WARN EPEERINVALID<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN EPEERINVALID<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN EPEERINVALID<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN EPEERINVALID<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN EPEERINVALID<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN EPEERINVALID<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.
npm WARN EPEERINVALID<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none was installed.

They were handled as peer dependencies in npm 2 – there's a bunch of things not working correctly with npm 3 and Steroids CLI currently, see https://github.com/AppGyver/steroids/issues/1090
Until we fix this, the sure-fire solution is to use Node 0.12 and npm 2 (nvm comes in handy). Sorry for the inconvenience.


Adding pre-submit image build

Dependency for pre-submit testing PRs: https://github.com/nephio-project/nephio/issues/373

/approve

/lgtm

/assign @tliron @henderiw

/lgtm
/approve


adding ef to a ef core library

Hi all,I trying to rebuild a project I created with rc1, and I started trying to recreate a library with a ef context and a few api controllers to be shared among project.
To do this I created a .Net Core Class Library, copied a few classes, and opened project.json and tried to add the references to entityframework packages.
I tried to add EntityFramework.Core
{
  "version": "1.0.0-*",

  "dependencies": {
    "NETStandard.Library": "1.5.0-rc2-24027",
    "Microsoft.EntityFrameworkCore.SqlServer": "1.0.0-rc2-final"
  },

  "frameworks": {
    "netstandard1.5": {
      "imports": "dnxcore50"
    }
  }
}

and I have the error

Where can I find informations on how to create a library with ef?

You will need additional imports. See issue #5176

In short have to wait for rtm? I don't understand what I have to add

You need to add imports as mentioned here https://github.com/aspnet/EntityFramework/issues/5176#issuecomment-216705192

Closing as a duplicate of #5176. I have added an explanation of the workaround to our RC2 docs. (See https://github.com/aspnet/EntityFramework.Docs/pull/167).


change to use algorithm.Time in Algorithm Manager

Per issue 145, change TradeBars to use algorithm.Time instead of UTC.
Mark DataDictionary Time as obsolete.

Thank you @bizcad !


What is the best way to clone, install and run this remotely on AWS?

Hi, this is an excellent example and I am fine running it on my local machine.
I would like to set it up myself remotely also on AWS. I have tried running a build from S3 but that causes a security issue on the web cam access. I am now attempting to clone into an EC2 instance and run dev from there, opening port 3001 and using the public url for the instance in my browser, but that just gives me a 'didn't send any response' error. Is there a quick fix for this?

Try adding host: '<IP_ADDRESS>' to the devServer config: https://github.com/awslabs/amazon-kinesis-video-streams-webrtc-sdk-js/blob/master/webpack.dev.config.js#L12-L16
See Webpack docs: https://webpack.js.org/configuration/dev-server/#devserverhost

Thanks! All I needed to do in addition was to add disableHostCheck: true to devserver, and it's up and running.


Unable to parse PDFs, "Failed to resolve 'test.elicit.org'"

When trying to run the "Loading paper text" chapter from the Primer, I run into an error indicating that it can't find "test.elicit.org". Since paper.parse_pdf depends on this remote resource to parse the PDF, it can't proceed at all.
Here's a full trace of what I see:
Full trace
python recipes/paper_hello.py --paper papers/keenan-2018.pdf
/home/cass/src/ice/venv/lib/python3.11/site-packages/pydantic/_migration.py:283: UserWarning: `pydantic.generics:GenericModel` has been moved to `pydantic.BaseModel`.
  warnings.warn(f'`{import_path}` has been moved to `{new_location}`.')
/home/cass/src/ice/venv/lib/python3.11/site-packages/pydantic/_internal/_config.py:334: UserWarning: Valid config keys have changed in V2:
* 'keep_untouched' has been renamed to 'ignored_types'
* 'fields' has been removed
  warnings.warn(message, UserWarning)
Traceback (most recent call last):
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/urllib3/connection.py", line 198, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/urllib3/util/connection.py", line 60, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/.pyenv/versions/3.11.0/lib/python3.11/socket.py", line 961, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
socket.gaierror: [Errno -2] Name or service not known

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 793, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 491, in _make_request
    raise new_e
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn
    conn.connect()
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/urllib3/connection.py", line 616, in connect
    self.sock = sock = self._new_conn()
                       ^^^^^^^^^^^^^^^^
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/urllib3/connection.py", line 205, in _new_conn
    raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x751bb09bb4d0>: Failed to resolve 'test.elicit.org' ([Errno -2] Name or service not known)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/requests/adapters.py", line 589, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 847, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='test.elicit.org', port=443): Max retries exceeded with url: /elicit-previews/james/oug-3083-support-parsing-arbitrary-pdfs-using/parse_pdf (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x751bb09bb4d0>: Failed to resolve 'test.elicit.org' ([Errno -2] Name or service not known)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cass/src/ice/recipes/paper_hello.py", line 10, in <module>
    recipe.main(answer_for_paper)
  File "/home/cass/src/ice/ice/recipe.py", line 176, in main
    defopt.run(
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/defopt.py", line 348, in run
    call = bind(
           ^^^^^
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/defopt.py", line 255, in bind
    call, rest = _bind_or_bind_known(*args, _known=False, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/defopt.py", line 203, in _bind_or_bind_known
    args, rest = parser.parse_args(argv), []
                 ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/.pyenv/versions/3.11.0/lib/python3.11/argparse.py", line 1862, in parse_args
    args, argv = self.parse_known_args(args, namespace)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/.pyenv/versions/3.11.0/lib/python3.11/argparse.py", line 1895, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/.pyenv/versions/3.11.0/lib/python3.11/argparse.py", line 2103, in _parse_known_args
    start_index = consume_optional(start_index)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/.pyenv/versions/3.11.0/lib/python3.11/argparse.py", line 2043, in consume_optional
    take_action(action, args, option_string)
  File "/home/cass/.pyenv/versions/3.11.0/lib/python3.11/argparse.py", line 1955, in take_action
    argument_values = self._get_values(action, argument_strings)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/.pyenv/versions/3.11.0/lib/python3.11/argparse.py", line 2485, in _get_values
    value = self._get_value(action, arg_string)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/.pyenv/versions/3.11.0/lib/python3.11/argparse.py", line 2518, in _get_value
    result = type_func(arg_string)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/src/ice/ice/recipe.py", line 181, in <lambda>
    Paper: lambda path: Paper.load(Path(path)),
                        ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/src/ice/ice/paper.py", line 158, in load
    paragraph_dicts = parse_pdf(file)
                      ^^^^^^^^^^^^^^^
  File "/home/cass/src/ice/ice/cache.py", line 28, in sync_wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/src/ice/ice/paper.py", line 119, in parse_pdf
    r = requests.post(PDF_PARSER_URL, files=files)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/cass/src/ice/venv/lib/python3.11/site-packages/requests/adapters.py", line 622, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='test.elicit.org', port=443): Max retries exceeded with url: /elicit-previews/james/oug-3083-support-parsing-arbitrary-pdfs-using/parse_pdf (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x751bb09bb4d0>: Failed to resolve 'test.elicit.org' ([Errno -2] Name or service not known)"))

❓ Is there an alternative that folks recommend for PDF parsing here?

I have quick-fixed this here https://github.com/TommyBark/ice/tree/fix-parse_pdf by using pdfminer.six package.
The semantic chunking is not very reliable as it is done based on html parsing and not all pdfs work nicely with it, but it works as proof of concept for the Factored Cognition Primer examples.


enable support for<EMAIL_ADDRESS>and above?

Describe the solution you'd like*
enable the package to support for<EMAIL_ADDRESS>version and above
currently when install the @near-wallet-selector/core package it will throw an error
└─▪ npm install @near-wallet-selector/core -S
npm ERR! code ERESOLVE
npm ERR! ERESOLVE unable to resolve dependency tree
npm ERR!
npm ERR! While resolving<EMAIL_ADDRESS>npm ERR! Found<EMAIL_ADDRESS>npm ERR! node_modules/near-api-js
npm ERR<EMAIL_ADDRESS>from the root project
npm ERR!
npm ERR! Could not resolve dependency:
npm ERR! peer<EMAIL_ADDRESS>|| ^1.0.0" from<EMAIL_ADDRESS>npm ERR! node_modules/@near-wallet-selector/core
npm ERR!   @near-wallet-selector/core@"*" from the root project
npm ERR!
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.

Hi, @sweetim thank you for raising this issue.
From our understanding, the goal with the newest version(s) of near-api-js is to split this package into multiple small packages we will work on this one but currently, there's no documentation on how to properly use these "separate" packages.
In the meantime, you can still use the v1.1.0 of near-api-js in your dApp.
yarn add<EMAIL_ADDRESS>or npm install<EMAIL_ADDRESS>If there are any errors or bugs with the v1.1.0 of NAJ please let us know.

Hi @sweetim and thanks for raising this issue! I will close it for now, and if you have any additional questions feel free to reopen it.


MetadataBuilder class to provide a DSL for building metadata at runtime

Splitting discussion from #14 into separate discussions. Paging @dig412.
Is the main benefit of a MetadataBuilder to give a saner error checking during metadata construction?

Yeah, the extra error checking potential is good, but this kind of goes hand-in-hand with #16  - I can use MetadataBuilder in the createMeta method so that all the configuration for how an object persists is contained in that Mapper.
Example:
class MetaBuilder {
	public function setTable($table);
	public function setClass($class);
	public function setPrimaryKey($primaryKey);
	public function addField($fieldName, $columnName = null, $type = null, $readOnly = false, $setter = null, $getter = null);
	public function addOneToOneRelation($fieldName, $remoteClass, $keyField, $remoteKeyField = null);
	public function addOneToManyRelation($fieldName, $remoteClass, $keyField, $remoteKeyField = null);
	public function addAssociativeRelation($fieldName, $remoteClass, $linkClass);
}

$builder = new MetaBuilder();

$builder->setTable("event");
$builder->setClass("Event");
$builder->setPrimaryKey("id");
$builder->addField("id", "eventid", "autoinc");
$builder->addField("name", "eventname");
$builder->addField("startdate", null, "date");
$builder->addField("enddate",   null, "date");
$builder->addField("description", "notes");
$builder->addField("venueId");
$builder->addField("price");
$builder->addOneToOneRelation("venue", "Venue", "venueId");
$builder->addAssociativeRelation("artists", "Artist", "ArtistEvent");

It's starting to seem like I just put little DSLs everywhere I can...

I don't think this is philosophically aligned with Amiss's core aims of avoiding runtime overhead. I think perhaps a static validator that can be used in project tests is a better way to go.


How to add information to span spinner

Hi! This is a really awesome crate, but I'm unsure how to add information, not necesarily change style.
let's say I have this generic function
  #[tracing::instrument]
    fn test() {
        std::thread::sleep(std::time::Duration::from_secs(60));
    }

All I get in my terminal is:

(not sure how to call this, so I called it "span spinner" in title!)
I would like to add file and line info.
I thought something like this would do it:
let indicatif = IndicatifLayer::new();
let indicatif_layer = tracing_subscriber::fmt::layer()
    .with_file(true)
    .with_line_number(true)
    .with_writer(indicatif.get_stderr_writer())
    .with_filter(LevelFilter::INFO);

But it makes no difference to the output

I have no clue what you're up to. Are you cobbling together code with LLMs?

Haha, I'm not really sure what you are asking.
let me make you a minimal reproducible example:
Let's have this as our setup for both cases:
use tracing_subscriber::{layer::SubscriberExt as _, util::SubscriberInitExt as _};

#[tracing::instrument]
fn test() {
    #[tracing::instrument]
    fn test() {
        std::thread::sleep(std::time::Duration::from_secs(1));
    }

    test();

    test();
}

Now in a typical example with tracing and pretty:
fn main() {
    tracing_subscriber::registry()
        .with(
            tracing_subscriber::fmt::layer()
                .with_span_events(FmtSpan::FULL)
                .pretty(),
        )
        .init();

    test();
}

There is file and line information available in the spans when they are printed:

(see main.rs:7)
I was wondering if this information is available in tracing indicatif, and how to make it visible if so, because with tracing_indicatif:
fn main() {
    let indicatif = tracing_indicatif::IndicatifLayer::new();
    let indicatif_layer =
        tracing_subscriber::fmt::layer().with_writer(indicatif.get_stderr_writer());

    tracing_subscriber::registry()
        .with(indicatif_layer)
        .with(indicatif)
        .init();

    test();
}

All I see is this:

Okay, but I have never used tracing-indicatif and am completely unfamiliar with what it does/how it works.
Did you mean to file an issue against it?


Update to Go 1.16.2.
Remove some unused tools.
Fix some build issues on MacOS.
Add 'osusergo' tag to remove some link-time warnings.
Remove 'demo.json' file, not used in Scope.
Miscelaneous other build improvements.

Great work - this looks much cleaner now!


Update makefile

Proposed changes
Jira ticket: CLOUDP-114366

Note: This only happens for developers.

After releasing mongocli/v1.23.0, builder will not be able to process current version since semver will throw an error while trying to parse version.Version in builder.go.

Version translation

Checklist

[x] I have signed the MongoDB CLA
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have added any necessary documentation in document requirements section listed in CONTRIBUTING.md (if appropriate)
[x] I have addressed the @mongodb/docs-cloud-team comments (if appropriate)
[x] I have updated e2e/E2E-TESTS.md (if an e2e test has been added)
[x] I have run make fmt and formatted my code

Further comments

@gssbzn I have this ticket https://jira.mongodb.org/browse/CLOUDP-114080 to fix that issue. This issue is still here even if we don’t merge this change.


Request: AJAX- specify content to use in Modaal

When calling content to the Modaal via AJAX, it would be really beneficial to specify a content area to use, rather than using the whole page and having to manually remove unnecessary markup.
jQuery supports this kind of function through it's .load function (a shorthand of .ajax)
$( "#result" ).load( "ajax/test.html #container" );
This allows the targeted page to be a 'complete' page, so if javascript fails or is disabled, or even if you wanted to disable the functionality of mobile for example, the user could still go to a legit page with regular markup (headers and footers etc...).
Love the Modaal by the way.

Here is a little quick and dirty solution i added in the fetch_ajax function;
                    // content fetch is successful so push it into markup
                    var target = $('#' + self.scope.id).find('.modaal-content-container');
                    if ( target.length > 0){

                        var content = data;

                        //see if there is a containing section to pull the content from, rather than the whole page
                        var contentArea = $(data).find('.modaal-ajax-content-area');

                        if(contentArea.length > 0) {
                            content = contentArea;
                        }

                        target.removeClass( self.options.loading_class );
                        target.html( content );

                        self.options.ajax_success.call(self, target);
                    }

Then all the user needs to do is wrap the content of the page being requested in a class, modaal-ajax-content-area and the AJAX function will pull that out, rather than the content of the entire page. Totally optional.

hey @alexborton, thanks for opening the ticket, and the comment re Modaal, we're glad we can provide something useful to other developers!
Am I understanding correctly that what you're proposing is we evolve the fetch_ajax function so that we're only returning a section of the overall file?
Example: my_file.php has the following markup

... head stuff here..

Element 1 content

Element 2 content

Element 3 content

And you'd like to only ajax in the content for  #element-2 into your Modaal instance?
If so, I can absolutely see the value in this and would be worth the investment to provide a flexible ajax option.

Yep, exactly that!
Whilst this is (probably) possible from the available events with the plugin, i feel it would be beneficial to have this built into the core.
The code i posted in my previous comment works correctly based on a pre-defined class to search for, if that class does not exist, it returns the whole document as it does/did by default. It probably requires some refinement to make sure multiple classes are not found, for example.

Ok awesome, thanks @alexborton, we'll put it down for something to investigate and address as I agree and think it's a pretty valuable use case.

+++1 on this one.  I've had to modify the script in the past, like @alexborton did.

+1 👍

Hey there @alexborton have you attempted to use your fix outside of editing the modaal.js file itself? I was trying to work out a way to extend it, but running into a wall.

I realized that the ajax_success event was never getting called. My only thought is that $.ajax stops executing when you set .html().
Because we are talking about modifying data, I added data to the parameters passed to ajax_success and moved the call to ajax_success up before target.html( data ); and set data equal to the value returned by ajax_success.
like this:
var target = $('#' + self.scope.id).find('.modaal-content-container');
if ( target.length > 0){
    target.removeClass( self.options.loading_class );

    data = self.options.ajax_success.call(self, target, data);

    target.html( data );
}

then in my configuration object I do this:
ajax_success: function(target, data) {
    return $(data).find('.selector');
}

Then I modified the default value of ajax_success to:
ajax_success : function(target, data){ return data; },

So we don't lose the data return in the AJAX response if we use a custom ajax_success method.

@danhumaan You marked this as assigned to yourself almost a year ago. Any plan on it making it to a release anytime soon? v0.4.5 shows 0% complete, which makes us sad

How do we integrate this in its entirety then?


[doc]  Remove unused config in record-level expire

Purpose
In https://github.com/apache/paimon/pull/4458 removed the config record-level.time-field-type, update doc.
Tests

API and Format

Documentation

+1


Cluster-autoscaler: use listers from ListersRegistry

Fix after #2226.
cc: @mumoshu @fgrzadkowski @jszczepkowski @MaciekPytel

This change is

@mwielgus Sorry it seems to had been slipped in while I was fixing somewhere not covered by unit tests! LGTM.


DataTypes should support custom null values

UNHCR numbers use * to indicate redacted data. In practice I want these to be nulls.

I already built this. Ha.


Fused HQQ Quantization Gemm

@msaroufim
Fused int4 / fp16 Quant Matmul
Fused kernel that combines asymmetric dequantization and gemm. Useful primarily for compute-bound (M > 16) scenarios and not for memory-bound / inference scenarios.
The kernel fuses two ops:

Dequantization: upcasts u4 / s4 weights to float16 / bfloat16, followed by groupwise scaling and shifting by scales / zeropoints
GEMM: matmul on dequantized weights and activations.

Tested and benchmarked for HQQ but could theoretically be used for any asymmetric quantization scheme.
Implementation Details

Bitpacking is simple row interleave, no need for extensive preprocessing (e.g., tinygemm or fastertransformer)
Tested for float16 / bfloat16 activations, scales, and zeros
Autotuned for both compute-bound and memory-bound configs
Assumes operand B of the gemm is is the quantized type.
Requires quantization along in-features, i.e., the K dimension, or axis=1, of torch.linear.weight.
Implementation handles both transposed and non-tranposed quantized weights, useful for forward / backward training passes.

Performance
Initial benchmarking (on A6000) demonstrates promising results, scaling well for compute-bound workloads:

M
N
K
group_size
dtype
hqq_ref
triton
tinygemm

0
16
4096
4096
128
torch.bfloat16
0.2675
0.0633
0.0382

1
32
4096
4096
128
torch.bfloat16
0.2669
0.0704
0.0649

2
128
4096
4096
128
torch.bfloat16
0.2689
0.0960
0.2523

3
256
4096
4096
128
torch.bfloat16
0.3268
0.1355
0.5192

4
512
4096
4096
128
torch.bfloat16
0.3628
0.2369
1.0892

5
1024
4096
4096
128
torch.bfloat16
0.5133
0.4753
2.2016

Times are in ms, see benchmarks/benchmark_hqq.py.
hqq_ref is the base HQQ_Linear module that is unfused (dequantization followed by call to torch.matmul).
tinygemm calls torch.ops.aten._weight_int4pack_mm. Implementation is a custom HQQLinear layer that wraps the preprocessing necessary for this kernel, adapted from a benchmark script posted by @mobicham from CUDA-mode Discord discussions.

GPU details:
_CudaDeviceProperties(name='NVIDIA RTX A6000', major=8, minor=6, total_memory=48676MB, multi_processor_count=84)

NOTE
This implementation requires triton >= 3.0.0.

Running tests / benchmarks requires installation of hqq:
pip install hqq

TODO

[x] clean up / refactor test to use pytest
[x] add transposed matmul
[ ] torch.compile benchmarking
[ ] introduce additional prologue / epilogue fusions
[ ] create standalone dequant kernel
[ ] test across range of gpus
[ ] adapt for other quant methods (other than hqq)

@msaroufim

Added a transpose implementation for backwards passes
Cleaned up test

skips unless triton >= 3.0 + hqq
refactored to use pytest
added transpose test
removed colored outputs

Benchmark

cleaned up imports
removed colors
print benchmark results to terminal rather than saving as csv

Attached my pip env (see reply above).


fix query timeout was not correctly set

Proposed changes

In the past, in ConnectContext's checkTimeout, if a timeout occurred and cancel was not passed PPlanFragmentCancelReason, it resulted in being set to Types.PPlanFragmentCancelReason.USER_CANCEL.
    public void cancel() {
        cancel(Types.PPlanFragmentCancelReason.USER_CANCEL);
    }

2024-03-29 13:11:11,683 WARN (connect-scheduler-check-timer-0|103) [ConnectContext.checkTimeout():921] kill query timeout, remote: <IP_ADDRESS>:50876, query timeout: 600000
2024-03-29 13:11:11,683 WARN (connect-scheduler-check-timer-0|103) [ConnectContext.kill():877] kill query from <IP_ADDRESS>:50876, kill mysql connection: false
2024-03-29 13:10:39,686 WARN (connect-scheduler-check-timer-0|103) [Coordinator.cancel():1443] Cancel execution of query a82b68a3a1bc44b9-b4bc0254ac412006, this is a outside invoke
2024-03-29 13:10:39,698 WARN (mysql-nio-pool-67|868) [ResultReceiver.getNext():102] Query a82b68a3a1bc44b9-b4bc0254ac412006 get result timeout, get result duration 599 ms
2024-03-29 13:10:39,698 WARN (mysql-nio-pool-67|868) [ResultReceiver.updateCancelReason():194] Query a82b68a3a1bc44b9-b4bc0254ac412006 already has cancel reason: USER_CANCEL, new reason fetch data timeout will be ignored

The ResultReceiver was canceled before getNext was called. The original code only set status.setStatus(Status.CANCELLED);, so there was no way to report a timeout error, only CANCELLED.
2024-04-09 08:39:14,106 INFO (mysql-nio-pool-31|1394) [Coordinator.execInternal():681] dispatch result sink of query bb9431afba254d2f-81b33c36f9dc136e to TNetworkAddress(hostname:<IP_ADDRESS>, port:9060)
2024-04-09 08:49:14,899 WARN (connect-scheduler-check-timer-0|102) [Coordinator.cancel():1443] Cancel execution of query bb9431afba254d2f-81b33c36f9dc136e, this is a outside invoke, cancelReason TIMEOUT
2024-04-09 08:49:14,900 WARN (connect-scheduler-check-timer-0|102) [ResultReceiver.cancel():214] ResultReceiver of query bb9431afba254d2f-81b33c36f9dc136e cancel failed, typically means the future is finished
2024-04-09 08:49:15,952 WARN (mysql-nio-pool-31|1394) [Coordinator.getNext():1303] Query bb9431afba254d2f-81b33c36f9dc136e coordinator get next fail, Cancelled, need cancel.
2024-04-09 08:49:15,952 WARN (mysql-nio-pool-31|1394) [StmtExecutor.sendResult():1721] cancel fragment query_id:bb9431afba254d2f-81b33c36f9dc136e cause errCode = 2, detailMessage = Cancelled
2024-04-09 08:49:18,818 WARN (thrift-server-pool-34|1681) [QeProcessorImpl.reportExecStatus():226] ReportExecStatus() runtime error, query bb9431afba254d2f-81b33c36f9dc136e with type SELECT does not exist

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall

run buildall


[scan][trainer] hotfix for xcresulttool spec changes >= XC16b3

Checklist

[x] I've run bundle exec rspec from the root directory to see all new and existing tests pass

for trainer module, all passed

[x] I've followed the fastlane code style and run bundle exec rubocop -a to ensure the code style is valid
[ ] I see several green ci/circleci builds in the "All checks have passed" section of my PR (connect CircleCI to GitHub if not)

PR tests on CircleCI is now broken due to unsupported version of Ruby; Will check when fix PR merged

[x] I've read the Contribution Guidelines
[ ] I've updated the documentation if necessary.

not sure if neccessary

[ ] I've added or updated relevant unit tests.

seems N/A to add test because mended codes never return any value externally (command execution only)

Motivation and Context

Resolves https://github.com/fastlane/fastlane/issues/22132
From Xcode 16 beta 3, fastlane ios scan started to fail after xctest had finished.
The reason is: There has been a deprecation and stderr in traditional xcrun xcresulttool command usage.

 Error details of xcresulttool related portion
...
00:07:00 Error: This command is deprecated and will be removed in a future release, --legacy flag is required to use it.
00:07:00 Usage: xcresulttool get object [--legacy] --path <path> [--id <id>] [--version <version>] [--format <format>]
00:07:00   See 'xcresulttool get object --help' for more information.
...

I tried new way when obtaining xcresult object (xcresulttool get tests or something) but

test summary was separated
the entire format of the object was totally different
test ID for --id option was not showing up in the report

For these reasons, decided to add --legacy if using Xcode 16 beta 3.
It would take quite a time to use the new type of xcresulttool like in the deprecation notice.
Description

used changes of this branch as a fastlane gem and tried run_tests under XC15 and 16.
specs in trainer had nothing to test with this part so I tried scan on both Xcode 15.4 and 16 beta 3
Testing Steps

00:00:16 $ gem uninstall -Iax fastlane
00:00:16 git clone --single-branch -b dokimyj-patch-1 https://github.com/dokimyj/fastlane fastlane-dokimyj
00:00:16 bundle install --path=vendor/bundle

Both environments are in CI, I cloned this single branch and bundle install to use bundle fastlane.
00:06:53 ▸ Testing passed on 'iPhone 15 Plus'
00:06:53 ▸ Test Succeeded
00:06:57 Skipping HTML... only available with `xcodebuild_formatter: 'xcpretty'` right now
00:06:57 Your 'xcodebuild_formatter' doesn't support these 'output_types'. Change your 'output_types' to prevent these warnings from showing...
00:06:57 +-------------------------+
00:06:57 |      Test Results       |
00:06:57 +--------------------+----+
00:06:57 | Number of tests    | 12 |
00:06:57 | Number of failures | 0  |
00:06:57 +--------------------+----+

scan with Xcode 16 beta 3 using this branch
00:08:36 ▸ Testing passed on 'iPhone 15 Plus'
00:08:36 ▸ Test Succeeded
00:08:41 Skipping HTML... only available with `xcodebuild_formatter: 'xcpretty'` right now
00:08:41 Your 'xcodebuild_formatter' doesn't support these 'output_types'. Change your 'output_types' to prevent these warnings from showing...
00:08:42 +-------------------------+
00:08:42 |      Test Results       |
00:08:42 +--------------------+----+
00:08:42 | Number of tests    | 12 |
00:08:42 | Number of failures | 0  |
00:08:42 +--------------------+----+

scan with Xcode 15.4 using this branch
Seems no differences in the result!

I saw the spec implementation; writing tests for these changes is difficult, so we may skip writing specs.

Amended along with the review comments:

Separated method for xcrun xcresulttool get command generation (for Rspec test :) )
Judging from xcresulttool version, not xcrun --version

Reviewer recommended format version condition, but from Xcode 16 beta it has never changed in beta 3.

Constructing argument with %W instead of oneliner string
Added Rspec, formatted
Some writing style fixes

How I checked if the fixes work:
I tried on my local machine using Xcode 15.4, Xcode 16 beta, and Xcode 16 beta 3.

Xcode 15.4

$ DEVELOPER_DIR=/Applications/Xcode_15.4.app bundle exec fastlane ios test
[✔] 🚀

...

| xcode_path                                     | /Applications/Xcode_15.4.app                                      |
+------------------------------------------------+-------------------------------------------------------------------+

...

[09:55:36]: ▸ Test Succeeded
+------------------------+
|      Test Results      |
+--------------------+---+
| Number of tests    | 6 |
| Number of failures | 0 |
+--------------------+---+

...

Xcode 16 beta

$ DEVELOPER_DIR=/Applications/Xcode_16_beta.app bundle exec fastlane ios test
[✔] 🚀

...

| xcode_path                                     | /Applications/Xcode_16_beta.app                                   |
+------------------------------------------------+-------------------------------------------------------------------+

...

[09:59:38]: ▸ Test Succeeded
+------------------------+
|      Test Results      |
+--------------------+---+
| Number of tests    | 6 |
| Number of failures | 0 |
+--------------------+---+
...

Xcode 16 beta 3

$ DEVELOPER_DIR=/Applications/Xcode_16_beta_3.app bundle exec fastlane ios test
[✔] 🚀

...

| xcode_path                                     | /Applications/Xcode_16_beta_3.app                                 |
+------------------------------------------------+-------------------------------------------------------------------+

...

[10:18:47]: ▸ Test Succeeded
+------------------------+
|      Test Results      |
+--------------------+---+
| Number of tests    | 6 |
| Number of failures | 0 |
+--------------------+---+

...

And... would anyone please restart appveyor/pr to see if CI is green? I got no permission to start that job I think...

Great work! Thank you!


Unable to run Azurite in background

Which service(blob, file, queue, table) does this issue concern?
all
Which version of the Azurite was used?
3.22.0
Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)
installed with VS
What problem was encountered?
Azurite does not run in a background
Steps to reproduce the issue?
Start /B Azurite
We used to be able to run Azure Storage Emulator as service. Now it is killed and we need to use Azurite. But I can't run it in the background or hidden. Console window needs to hang about for it to work, no matter what I was trying.
I don't want to run docker container, as I don't have it running all the time.  Advice from another issue start /B azurite still makes the console window to float about which is just annoying.
Please bring back the background option.

@trailmax
I see you have raised similar question in https://github.com/Azure/Azurite/issues/589, and I have answered you in https://github.com/Azure/Azurite/issues/589#issuecomment-1571436288


update docs links

Fixes non-example.conf links in #3

Separating out fix for ../cmd/trickster/conf/example.conf update link.


Create edmonds15.txt

Edmonds School District has 4 high schools, all of which include computer science and programming classes. Mountlake Terrace High School also has a 4 year STEM program which also includes computer science classes<EMAIL_ADDRESS>is given to students in the district.
Here is a link to the district website:
https://edmonds.wednet.edu/
Here is a link saying that @edmonds15.org is given to students in the district:
https://www.edmonds.wednet.edu/cms/One.aspx?portalId=306754&pageId=489178#access

@Chqrix Pull request merged. Thank you!


Write a TypeScript program to convert seconds to hours

Description
Write a TypeScript program to convert seconds to hours
1 hour = 60 minutes = 3600 seconds
Input  : 18000
Output : 5

How to contribute

Save the solution in program/ConvertSecondsToHours.ts file
Add ConvertSecondsToHours.ts file in convert-seconds-to-hours folder

!assign


Refactoring pass2

get rid of all TODOs

HUGE refactoring is required:

https://github.com/Esentiel/HurricaneEngine3D/issues/38
Move input from rendering, and probably move window from rendering as well?
Remove all todos in code, make code clean
*if there will be time - https://github.com/Esentiel/HurricaneEngine3D/issues/41


TypeError: Cannot read property '$ref' of undefined when transfer openapi v3 to swagger v2

when execute the command :  api-spec-converter openapi-swagger.json --from=openapi_3 --to=swagger_2 > swagger-v2.json
got an error
Fatal TypeError: Cannot read property '$ref' of undefined

Are you able to share your input document via a gist etc?

@MikeRalphson  something like :
schema:
$ref: '#/components/schemas/User'

@nsw That's really not enough to go on. $refs are handled by the converter ok.

@MikeRalphson For example, if I have a API apec which don't have an associated schema, it will generate this error: Fatal TypeError: Cannot read property '$ref' of undefined
{
"info": {
"title": "Nodejs",
"version": "1.0.0"
},
"openapi": "3.0.0",
"paths": {
"/articles": {
"get": {
"operationId": "ArticleController.getAll",
"responses": {
"200": {
"content": {
"application/json": {}
},
"description": "Successful response"
}
},
"summary": "Get all",
"tags": [
"Article"
]
}
}
}
}
Seems that this following kind of config is necessary ? otherwise it will cause en error...
schema:
$ref: '#/components/schemas/'

@MikeRalphson By the way i use version 2.7.6 ..

@nsw thanks for the minimal example, I was able to reproduce using that. Published a fix in 2.7.7

@bobby-brennan   Thanks for the quick fix ! Super efficient.  It works as expected now :) :)


trouble with deserializing an complicated ISerializable

Hi,
I have a complicated ISerializable object and when i try to deserialize with json.net i get:
": Unable to find a constructor to use for type BriefFiniteElementNet.ElementCollection"
But the ElementCollectionClass does have deserialization constructor with protected scope:
protected ElementCollection(SerializationInfo info, StreamingContext context)
here is my code:
 class Program
    {
        static void Main(string[] args)
        {
            ISerializable model = BriefFiniteElementNet.RandomStuff.GetRandomFrameModel(5);

            //serialize - deserialize with Json.NET - fail
            var json = Newtonsoft.Json.JsonConvert.SerializeObject(model);
            var reterievedObject = Newtonsoft.Json.JsonConvert.DeserializeObject<Model>(json);//throws exception

            //serialize - deserialize with Json.NET - does work
            var st = new MemoryStream();
            var formatter = new BinaryFormatter();
            formatter.Serialize(st, model);
            st.Position = 0;
            var reterievedObject2 = formatter.Deserialize(st);//works fine
        }
    }

the model object which is complicated is from this project:
https://brieffiniteelementnet.codeplex.com
Am i missing something?

It needs to be public.

sorry, can you please explain more?
What needs to be public?!

The constructor.

Thanks, that will probably fix the problem.
But i think all ISerializable classes do not necessarily have public constructors. So that would be good if you can handle constructors other than public (like protected or private) via reflection as .NET binary serialization do.
thanks again


Reimplement exception handling

[ ]   General Comment
 Stage 1 :+1:
 Stage 2 :-1:

This pull request is now managed with CodeReviewHub. Refresh.

Stage one :+1:

Stage 2 :-1:

Stage 2 :+1:

:shipit:


Configure New Staging Environment for Institution Training

User Story
Ensure new Staging environment is ready for PSIs to submit Institution and Student accounts/application and have them be processed without unnecessary blockages.
ACs

[ ] Bypass CRA check
[ ] Bypass SIN check

@Joshua-Lakusta
Guru finished a ticket which finalized the new staging environment. It is a completely clean database - everything has to be done from scratch (IDIR login still works according to what's configured in TEST, though).
They also have this environment configured the same way as the previous staging environment, so CRA and ESDC checks will be running as expected in PROD, i.e they are not bypassed.
Do we need to make changes?

@Closing because SIMS team members will action the SIN validations manually


Allow to define animation FPS

We more often think of animaton speed in terms of "frame-per-second" that "frame-duration".
Therefore, the user should be able to define the animation FPS. (frame-duration =  1 / FPS)

Resloved in 7eb14dd


Extension debugging: Debugger stops but doesn't show call stack

VSCode Version: alpha from 15-04
OS Version: Linux

Steps to Reproduce:

In the vscode repo open /vscode/extensions/php as folder
set a breakpoint on /src/features/hoverProvider.ts, line 13 (in provideHover)
F5 to run the extension
in the dev target open a PHP file and hover over a word
Debugger stops, but call stack stays empty

This works in 1.0.0

VSCode is getting an empty callstack from node debug, forwarding to @weinand
I can also debug our nodeDebug next week to drill deeper.

Asigning to me to investigate

The error that the node adapter gets from node is
message:"Internal error: illegal access"
request_seq:25
seq:155
success:false
type:"response"
__proto__:Object

It happens here
Not sure if related https://github.com/node-inspector/node-inspector/issues/413
So it seems that the adapter has no access to the new node running as part of the extension host.
Will continue investigating, pasting info here maybe @weinand, @bpasero  have a suggestion

I did not hear about this issue before. Unrelated: You should really find a way to return these errors back to the adapter and display them somehow, the error in https://github.com/Microsoft/vscode-node-debug/blob/master/src/node/nodeDebug.ts#L1527 seems totally not being used?

Can not reproduce this with a small node 5.10 attach sample. So somehow trying to attach to node inside extension host makes the difference here.
I do not believe this is important since this will not get surfaced to the users. Removing that tag, @aeschli let me know if you disagree.

Fixed this morning by Ben and me


RFE: Optionally log page that is being processed

Use case: I am trying to publish a lot of AsciiDoc pages, but many of them have significant quality issues. If the Confluence Publisher is unable to process an AsciiDoc page, I'd like to know so I can check it out, and fix the issue.
Current situation: Currently, when an exception is thrown, it is not clear which page (AsciiDoc file) is considered problematic. Additionally, it does not seem to be possible to enable logging so that each AsciiDoc file that starts being processed is being logged.
Request:

When there is an exception, please log the name and/or path of the AsciiDoc file that is being processed.
Support an option/parameter to log the name and/or path of each AsciiDoc file as it is being processed.

Hi @znerd, thank you for raising this feature request. I agree that the logging in case of conversion issues is somewhat "minimal" (e.g. a broken image reference is logged via the exception, but without any context).
To better understand your situation and to have real-world examples for testing/verification: which type of quality issues do you see in your AsciiDoc files that lead to conversion failures (e.g. broken image references, broken page references, ...)?
I could imagine something similar to the ConfluencePublisherListener, but for the converter, providing callbacks for each file being converted, that could then be implemented to log via the Maven logger or to the console (for Docker), probably enabled via a debug option. Would that make sense in your use case?

Hi @cstettler : Yes, that makes a lot of sense. I will need some time (due to other priorities) to gather a reproducible test case.


add werkzeug routing: allows '/endpoint/<some_id>' style routing

This adds support for variable url paths just like with Flask's app.route decorator, so you can now do things like:
@sockets.route('/submit/<game_id>')
def submit(ws, game_id):
    ...

I hadn't seen pull requests #7 and #14 before I wrote this, this is essentially the same thing but done differently (and with less changes) so maybe you'll like this better? I used werkzeug since this is what Flask is using. This works fine for me although I only tested it in the context of my project. If you think this doesn't make sense I'm happy to take suggestions and improve it, seems like this is a feature a lot of people want.

:+1:
I'm also interested in seeing that merged it.

Any chance of getting this released any time soon?

The above is working great for me, especially in tandem with the other PRs here. BTW, @kennethreitz (or anyone else who wants an updated package), I've merged all the outstanding PRs together in my master over here: https://github.com/philipn/flask-sockets

:sparkles: :cake: :sparkles:

Sorry for the long delay on this — I'm really excited about it :)


Release 0.5

Changes Unknown when pulling 4a0b807b263eb05df20c1142b5ab65f6fb003f8b on release-0.5 into ** on master**.

Changes Unknown when pulling c54d616e2e3784dca5facb16aebd9dfbd7b10675 on release-0.5 into ** on master**.

Changes Unknown when pulling 61b625b7a7a29715282c1ca1c95e06316ba32342 on release-0.5 into ** on master**.

Changes Unknown when pulling 8fdd427ab9a2bb90bd4a4af9de6c845d4f8aba44 on release-0.5 into ** on master**.

Changes Unknown when pulling 8fdd427ab9a2bb90bd4a4af9de6c845d4f8aba44 on release-0.5 into ** on master**.

Changes Unknown when pulling 9490c3556b1ba3bdc464b43e3013a0e25d76eb8f on release-0.5 into ** on master**.

Changes Unknown when pulling 4a0b807b263eb05df20c1142b5ab65f6fb003f8b on release-0.5 into ** on master**.

Changes Unknown when pulling ad3ef47181ba68b35a41e1e51485ceeb1557c7bf on release-0.5 into ** on master**.

Changes Unknown when pulling ad3ef47181ba68b35a41e1e51485ceeb1557c7bf on release-0.5 into ** on master**.

Changes Unknown when pulling ad3ef47181ba68b35a41e1e51485ceeb1557c7bf on release-0.5 into ** on master**.

Changes Unknown when pulling ad3ef47181ba68b35a41e1e51485ceeb1557c7bf on release-0.5 into ** on master**.

Changes Unknown when pulling 9a4ffecec8556da1749022a8754f7f46e8512f01 on release-0.5 into ** on master**.

Changes Unknown when pulling 9a4ffecec8556da1749022a8754f7f46e8512f01 on release-0.5 into ** on master**.

Changes Unknown when pulling d2ec1029cb2cd3bdfa3d23c170d1a65eade58449 on release-0.5 into ** on master**.

Changes Unknown when pulling d2ec1029cb2cd3bdfa3d23c170d1a65eade58449 on release-0.5 into ** on master**.

Changes Unknown when pulling d2ec1029cb2cd3bdfa3d23c170d1a65eade58449 on release-0.5 into ** on master**.

Changes Unknown when pulling 02d374b4098e96092d2bd2ea4512846226247724 on release-0.5 into ** on master**.


add arxiv:2309.17122

adding our paper arxiv:2309.17122 with title "Benchmarking the Abilities of Large Language Models for RDF Knowledge Graph Creation and Comprehension: How Well Do LLMs Speak Turtle?" as survey. It compares the ability of different LLMs for a couple of tasks.

Thanks for your contribution !


Support non-Linux kernels in NixOS

Nix itself supports many platforms including FreeBSD and Darwin, however NixOS is Linux exclusive. In the olden days, if someone wanted to try out FreeBSD, they'd have to install the whole system and dual-boot, but with NixOS, it's conceivable that one would be able to install multiple kernels into the same OS installation and switch between them at boot. While Linux applications wouldn't work on, say, GNU HURD, the Nix store would allow multiple versions of the same application where each is compiled for a different kernel. If I'm not mistaken, this already happens for people who share their nix-store between OS installations.
Naturally, porting all of NixOS to alternate kernels would be a monumental task, but I would like to at least throw the idea out there if someone decides to start pursuing it.
Obviously, this would require removing all Linux specific utilities from the core of NixOS, or at least making them optional. The biggest dependency that I can think of is Systemd. Which would make the first step of this task be to make NixOS services agnostic to the init process.
Known tasks. (To be updated with more specifics.)

[ ] Semi-support https://github.com/InitWare/InitWare instead of Systemd

[ ] Add some sort of feature flag to strip of systemd-only features from unit files

[ ] Improve package compatibility across more systems.

[ ] Make sure all core utilities are available for desired kernel or can be replaced by one that is.

[ ] Provide build instructions for kernels other than Linux.

[ ] NetBSD

[x] builds
[ ] runs

[ ] GNU Hurd, Nixpkgs used to contain something for

[ ] Rig up booting. @Ericson2314 has no idea how to even break this down.
[ ] Provide configuration.nix switch for selecting kernels.
[ ] Some sort of QEMU smoke test.

(I'm certain the tasks should be broken up smaller, but this is what I can think of right now.)
What do you think? Is this worth pursuing? Is this even feasible?

Well, I have purposed it some time ago, as a Google Summer of Code project. We have an "unofficial" repository for GSOC ideas [1], and one of them was creating NixOS versions using kFreeBSD instead of Linux.
Also, there was a small discussion in the mail list about making NixOS init-agnostic (today we are strongly tied to systemd).
P.S.: As a sidestep project, I am working alone and slowly porting Nix to NetBSD.
1 - https://github.com/teh/gsoc

There are some attempts to abstract out some parts of services, but they always die out (partially because people wanting to use something other than systemd stop using NixOS, even if they still use Nix on some other system).
I think you should just use the existing functionality for converting systemd services to runner scripts until that becomes the biggest problem (hopefully at that point you will have more leverage and more understanding).
Adding build scripts for other kernels (probably some BSDs) sounds like something that is likely to be merged just because it is one more package with little global impact; and if turns out to be harder than you think, this part is a hard dependency of the entire project.
I am not completely sure what is your plan for the basic stdenv on top of BSD kernels, you need some minimal things even to boot. Do you have some cross-building plan in yout mind? Or do you want to bootstrap on real FreeBSD/OpenBSD/NetBSD installations and then update bootstrap tools in the same way as it happens on NixOS/Linux?
Once you have (2) and (2.1) being minimal non-Linux environment, you should be able to easily write some expressions generating minimal boot scripts. Services can be taken from NixOS via runner script export and also launched from the boot scripts. Then you will know what you need for integration.
My personal opinion is that the only thing that you want to modify on NixOS side is the bootloader config generation: currently NixOS assumes it is the only Nix-based OS in existence, which is already not completely literally true; if there was a way to have multiple profiles like system and to provide bootloader configuration parts that can be assembled together…

There are some attempts to abstract out some parts of services, but they always die out (partially because people wanting to use something other than systemd stop using NixOS, even if they still use Nix on some other system).

I suppose that's one advantage to doing it like this, as I don't care as much about systemd and it's inclusion in NixOS won't make me stop using it, but having the system tied to something like systemd means that it doesn't feel like Nix is reaching its full potential.
As for the bootstrap problem, one possibility could be to build upon the existing build-vm option of nixos-switch. That said, I initially imagined that getting a bootstrapped system would be simpler than getting the core environment to work on FreeBSD.

Well, maybe I would be more willing to consider extending NixOS if I could dual-boot it easier (testing X.org drver config in VM on a notebook is a bit too complicated for my liking).
As for taking stuff from NixOS, I currently use the following code (Linux-based system with packages only from NixPkgs, hand-written bootscripts):
{pkgs, nixos}:
{
  serviceScript = name: config:
    (builtins.getAttr name (nixos {configuration = config;}).config.systemd.services).runner;

  etcSelect = filename: config:
    let
      nixosInstance = nixos {configuration = config;};
      selected = (pkgs.lib.filterAttrs (k: v: v.target == filename) nixosInstance.config.environment.etc);
        source = (builtins.getAttr (builtins.head (builtins.attrNames selected)) selected).source;
    in (if pkgs.lib.isString source then source else source.outPath);
}

Unfortunately, some services neither put their configs in /etc/ not provide any way to access the config path from the outside via attributes.
As for bootstrap, I meant bootstrapping environment for not-currently-supported kernels.

This should be a project, not just an issue, because the task is huge.

This issue depends on this one: https://github.com/NixOS/nixpkgs/issues/26067

This is a very wishful post but I feel it's more likely and better if there were a BSD distro with systemd... I'd hate for systemd services from Nix to go away...

Systemd on non-Linux might be a pretty tough project by itself, I suspect.

Systemd on non-Linux might be a pretty tough project by itself, I suspect.

Yes, it is:
https://uglyman.kremlin.cc/gitweb/gitweb.cgi?p=systembsd.git

This is still important to me.

There's a new attempt to port systemd to BSD: https://github.com/InitWare/InitWare

There's a new attempt to port systemd to BSD: https://github.com/InitWare/InitWare

Is this ported to nixos?

There's a new attempt to port systemd to BSD: https://github.com/InitWare/InitWare

Is this ported to nixos?

I don't believe so, but AIUI the project has only been going for a
couple of weeks.

From original site:

The following platforms are supported:

DragonFly BSD (5.8+) as user manager.
FreeBSD (13.0+) as user manager.
GNU/Linux (3.6+) as system or user manager. n.b. GNU/Linux support is complete but not yet set up to build; this will be corrected later.
NetBSD (8.0+) as system or user manager.
We hope to support (recent versions of) OpenBSD, and possibly also Illumos, in the near future.

Looks promising!

Also note that the NetBSD kernel supports running Linux binaries natively, and can be booted by Grub. It might not be as trivial as a drop-in replacement of the Linux kernel to get it to effectively work, but this could be another path to solving this issue.

I am favourable to the idea of NixOS on a BSD kernel and would be pleased to contribute to that end with InitWare, though I would stress that InitWare is not stable yet, and I doubt I will be comfortable to recommend its use until it has matured significantly.
I should also note that InitWare is not and doesn't aim to be just "systemd for BSD" (and things like resolved and networkd are entirely outwith my skillset and time to deal with) but it does aim to retain compatibility with systemd unit-files, its D-Bus interfaces, the systemctl tool, etc to the fullest extent possible. But this should not pose a problem with a NixOS porting effort, as those interfaces which you use should be supported well enough to leave only a few unavoidable problems, inherent to the different platforms, to deal with.
Portability does not include the most highly Linux-specific things, but where possible I support these by alternative means, and where useful features exist elsewhere I support those on a per-platform basis. For example, one of the main merits of CGroups is the ability to track the processes of units reliably, even while they fork off other processes. This is achieved on InitWare in BSD ports with a lightweight CGroups-like abstraction layer over the PROC event filter for Kernel Queues, by which any process can be tracked (with forks and exits noted).
Thus one of the 'killer features' of systemd is supported fine on BSD. But note that this doesn't extend to the hierarchical resource limitation and accounting. That is one thing for which there is no straightforward alternative on all BSD platforms; I am not even sure whether FreeBSD's jails can be used to that end, as they are something more of an all-or-nothing affair rather than CGroups' total modularity.
InitWare is not then an a means to magically boot a GNU/Linux with systemd system in unmodified form with another kernel. But I think it does replace the more complicated task of adapting NixOS to a totally different service manager instead with a simple task of dealing with a few corner-cases and Linuxisms.

Also note that the NetBSD kernel supports running Linux binaries natively, and can be booted by Grub. It might not be as trivial as a drop-in replacement of the Linux kernel to get it to effectively work, but this could be another path to solving this issue.

NetBSD's Linux support is not yet adequate to run systemd. This is not however an impossible goal; I think systemd can run in an LX branded zone in Joyent's branch of Illumos, so the idea is not unthinkable.
That having been said, I can think immediately of at least 2 interfaces without which systemd-the-PID1 will not run in any useful capacity under NetBSD:

EPoll and its family of event sources (TimerFD, iNotify, SignalFD, etc.) These are used extensively by systemd. They are however present in FreeBSD's Linuxulator and now are supported natively by Illumos following their import from the Joyent branch.
CGroups, without which systemd may run, but without tracking of processes across forks, nor reliable determination of the unit to which a PID belongs (a functionality used extensively both within systemd and without by its API consumers.) This is also provided by LX BrandZ in Joyent's Illumos branch.

I am ignoring the question of stage-1 initialisation here (i.e. the low-level setup of the system, during which virtual filesystems are mounted and other such important things happen) because there are numerous intricacies which would have to be emulated if you would run an unmodified Linux build of systemd as PID 1 on NetBSD. On the other hand, sidestepping in favour of e.g. calling out to another binary is relatively straightforward and would not require huge modifications to systemd.

I think there is a question of merit when it comes to relying on a layer like NetBSD's compat_linux or FreeBSD's Linuxlator.
It's definitely the case that you can have a ~working system by immediately performing a chroot into a (old) (GNU/)Linux userspace after the system finishes booting. But you don't have access to any native functionality, so what you have is basically a system that's identical to Linux with whatever amount of functionality unavailable and a different set of device driver code (which is slightly good for system diversity, but not really). Applications cannot fully take advantage of the system's functionality because they don't know the system's functionality, as far as they're concerned they're running under a neutered Linux system.
In NetBSD, compat_linux isn't loaded by default because it isn't subject to the same rigorous testing as the normal system call interface, isn't as mature as the normal system call interface, and has been subject to a number of serious bugs as a consequence. There is a branch under development that adds support for the full Linux epoll interface, but it's specifically because a proprietary application started requiring it - compat_linux is fundamentally a last resort for running proprietary binaries. It's a white whale.

IMHO the idea is that the resulting system can be legitimately be called a "NetBSD distro", not relying in such things as compatibility layers (except maybe in bootstrap).

@netbsduser All those caveats sound find to me me. In fact, I would turn it around and say "initware focuses on the level of compat that is feasible and useful for an cross-distro journal" is useful for me. Basically, I hope we could end in a very synergistic situation where multi-kernel NixOS and initware are each greatly helped by the other.

Also re sandboxing, I am personally interested in things like capsicum / CloudABI that I believe are conceptually simpler and nice interfaces as better candidates for cross-kernel support than the blunt jails/zones/namespaces we have today. I am hoping this could push that along, for example:

FreeBSD actually has merged capsicum / CloudABI unless I'm mistaken
I think systemd/initware principles are broadly compatable with that so a port is possible
I think the biggest problem with capsicum / CloudABI was the userland, which a 'initware + capsec" could help bootstrap
🤞🤞🤞 If the above works out well, maybe it becomes worth someones while to dust of capsicum / CloudABI in linux.
🤞🤞🤞 while I think upstream systemd doesn't care about supporting non-Linux, they may care about supporting capsicum in Linux.
I they do that, initware systemd have a path to convergence after all.

This is all rather utopian, but i do believe it's faintly possible. I think capability-based security of this sort is the rare thing that:

is incremental, not reinventing the world all at once (especially if "cap land" can spawn conventional containers for "scoped" back compat)
Makes computing more secure
Makes programming easier

and that's rather compelling mix of high value relatively low cost.
To be clear, I still think kernel diversity is compelling on it's own. A major victory on it's own, even. I bring up the above because I think kernel diversity for NixOS is also the best way to bring it about :).

related: https://github.com/NixOS/nix/pull/3160

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/what-is-sandboxing-and-what-does-it-entail/15533/1

any updates on this? seems its still in limbo

any updates on this? seems its still in limbo

Well, among some things I am working on the samurai+muon replacements for ninja+meson, in order to make nix at least POSIX-portable+bootstrappable with your project.
On the other hand I am still interested in at least two exotic platforms: Haiku and NetBSD.

nice, sounds cool as heck :)

I am a NetBSD developer, do not hesitate to drop me a line if you need help with anything.

About this:
https://github.com/NixOS/nixpkgs/pull/169961

Synit is a another upcoming, kernel-independent init system. It's still experimental but partially packaged.

Still important for me.

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/is-nixbsd-a-posibility/29612/4

Still important for me.

work on https://github.com/NixOS/nix/pull/3160 is continuing again :) if you'd llike to help out let me know!

@khorben excuse me for the ping; what we need to do to nix evaluator into pkgsrc mainline?
@p01arst0rm do you think we should wait for the Meson Builds Nix be effectively implemented before pushing it to NetBSD pkgsrc?

@p01arst0rm do you think we should wait for the Meson Builds Nix be effectively implemented before pushing it to NetBSD pkgsrc?

i think meson builds nix will greatly increase portability making the process much easier. although, i have previouly committed compatibility updates for *BSD , so i dont see it being strictly required :)

@khorben excuse me for the ping; what we need to do to nix evaluator into pkgsrc mainline?

Sorry for the (very) late answer @AndersonTorres; the easiest ways to gain visibility from the pkgsrc community are:

Reaching out on the mailing-list<EMAIL_ADDRESS>(https://www.netbsd.org/mailinglists/#tech-pkg for instructions)
Pushing to the pkgsrc-wip Git repository (https://pkgsrc.org/wip/ and https://pkgsrc.org/wip/users/ for commit access)

Let me know if you need help with this!


feat(FieldTheory/Perfect): add rootsExpand[Pow]EquivRoots[_apply]

... which state that over a perfect integral domain R of characteristic p, x ↦ x ^ p ^ n is a bijection from the set of roots of Polynomial.expand R (p ^ n) f to the set of roots of f.

maintainer merge


Support externalize in the JSON format

What type of PR is this?
Improvement

What does this PR do / why is it needed ?
Allows users to specify the json format in externalize

Other notes for reviewers:
Without this feature, users will get below error when invoking ->externalize('application/json')
java.lang.UnsupportedOperationException: Externalize TDS not supported by format - application/json
	at org.finos.legend.engine.external.shared.runtime.ExternalFormatRuntimeExtension.executeExternalizeTDSExecutionNode(ExternalFormatRuntimeExtension.java:47)

Does this PR introduce a user-facing change?
No, this feature doesn't exist today

This PR is stale because it has been open for 30 days with no activity. Please remove stale label or add any comment to keep this open. Otherwise this will be closed in 5 days.


Ability to add custom LNC mailbox server to GUI

As mentioned in the LND devloper Slack:
I see in the docs here https://docs.lightning.engineering/lightning-network-tools/aperture/mailbox#docs-internal-guid-6d497483-7fff-ccdd-3290-061a74b72572 that once you setup your own LNC Mailbox with Aperture you then need to run a CLI command with litd litcli sessions add --label="My own mailbox" --type admin --mailboxserveraddr lnc.yourlightningapp:443 before you can use your own Mailbox to connect to applications. I think it would be good if the above command was exposed in the GUI of Terminal so that non-technical users can add custom Mailboxes. This would make it easier for more Mailbox providers to popup. For example if an LNC app has its own Mailbox, users can access it more easily than having to use the CLI which is a non-starter for a lot of people.

My issues seem related:

https://github.com/lightninglabs/lightning-terminal/issues/586
https://github.com/lightninglabs/lightning-terminal/issues/588

I found this in the GUI under Custom Permissions > Advanced Options when creating a new LNC connection. So it looks like this functionality is already possible? Although it is fairly hidden.

Yes, that feature is already present. There's always a tradeoff between a simple to use UI for everyone and having all the options for power users. So if you have an idea of how to make this more apparent to power users without confusing most other users, we're all ears. I personally always open all "advanced options" sections in all UIs since I know that's where the good stuff usually is hidden.
Closing the issue for now, feel free to re-open if you think this needs adjustment.


Add some more information about plugins

such as how its the 2nd step in babel's 3 steps for more context

K updated


Remove .cache dir from cloud sync

For storage path /neatlynx/test4, dvc sync data uploads files to  /neatlynx/test4/.cache directory.
.cache is obviously not needed.

We've discussed this issue with @dmpetrov and agreed to just leave it be for now. Closing.


Enable NVIDIA services in the bootc image

The nvidia-persistenced and nvidia-fabricmanager services should be started on machines with NVIDIA devices. Fabric Manager is only needed on machines with an NVLink switch, so we patch it to start only if /dev/nvswitchctl is present.

/cc @kwozyman

Looks okay to me


Interesting use case

Hi all,
I'm relatively new to istio and currently trying out some more complex scenarios and can't figure out from the documentation of this scenario is supported - and would really like guidance.
At a very high level, I have an istio deployment setup with mutual-tls so when I deploy pods with istioctl, the sidecar transparently handles the mtls for me, which is wonderful.
However, there are certain pods that I still wish to be part of this mesh (so we get the logs etc), but excluded from inbound mtls, IE ingress communication to them does not require the certificates but egress traffic to other internal services does.
What this would facilitate is services that can be exposed on unique ingress IPs (in GKE world, services defined with Type:  LoadBalancer, or kubernetes Ingress types), as the model of a single ingress service to the mesh, and then using SNI to separate to downstream services is not compatible for my use case.
It's hard to explain in words, so here is a simplistic diagram:

Anyone have any ideas?
Thanks
Karl

I suppose in summary, I want to be able to create multiple Ingress which are exposed on separate external IPs, rather than the IP of the istio-ingress service

Closing this issue as i realised its a docs repo!  Re-raised it here https://github.com/istio/istio/issues/1752


Undefined variable: "$white"

Getting the following error when navigating to http://localhost:3000/admin after rails admin installation:
Sass::SyntaxError in Rails_admin/main#dashboard
Showing /Users/ryanresella/.rvm/gems/ruby-1.9.3-p0/bundler/gems/rails_admin-1830d106f272/app/views/layouts/rails_admin/application.html.haml where line #8 raised:
Undefined variable: "$white".
(in /Users/ryanresella/.rvm/gems/ruby-1.9.3-p0/bundler/gems/rails_admin-1830d106f272/app/assets/stylesheets/rails_admin/imports.css.scss.erb)
Extracted source (around line #8):
5:     %meta{:content => "text/html; charset=utf-8", "http-equiv" => "Content-Type"}
6:     %meta{:content => "NONE,NOARCHIVE", :name => "robots"}
7:     = csrf_meta_tag
8:     = stylesheet_link_tag "rails_admin/rails_admin.css"
9:     = javascript_include_tag "rails_admin/rails_admin.js"
10:   %body.rails_admin{:style => "padding-top: 60px;"}
11:     .topbar-wrapper{:style => "z-index: 5;"}

So what's the fix for this?

This might be a coincidence.. but I was using the gem twitter-bootstrap-rails for the main Rails application.  I removed that gem and switched to bootstrap-sass which seemed to fix the problem.

Yep, i was also using  twitter-bootstrap-rails, and changing it for bootstrap-sass fixed this.
Thanks!

Looks like it happens only when you use bootstrap less. There's workaround. Install 'bootstrap-sass' and rename your main bootstrap.css.less to twitter_bootstrap.css.less

Me too, just removed twitter-bootstrap-rails and used bootstrap-sass and it worked! :relieved:


patch source plugin can introduce non-determinism

See original issue on GitLab
In GitLab by [Gitlab user @valentindavid] on Aug 6, 2018, 10:40
Summary
As in #527, patch can introduce non-determinism during staging. This happens when the patch contains new files.

In GitLab by [Gitlab user @valentindavid] on Aug 6, 2018, 10:41
marked this issue as related to #527

In GitLab by [Gitlab user @valentindavid] on Aug 6, 2018, 10:42
marked this issue as related to #541

In GitLab by [Gitlab user @valentindavid] on Aug 6, 2018, 10:42
marked this issue as related to #542

In GitLab by [Gitlab user @valentindavid] on Aug 6, 2018, 10:42
marked this issue as related to #543

In GitLab by [Gitlab user @valentindavid] on Aug 6, 2018, 10:42
marked this issue as related to #544

In GitLab by [Gitlab user @valentindavid] on Aug 6, 2018, 10:42
marked this issue as related to #545

In GitLab by [Gitlab user @valentindavid] on Aug 6, 2018, 10:42
assigned to [Gitlab user @valentindavid]

In GitLab by [Gitlab user @valentindavid] on Aug 6, 2018, 17:09
mentioned in commit 4d142206bdba11bcef52e54c8c2ea4e1ad525d03

In GitLab by [Gitlab user @valentindavid] on Aug 7, 2018, 17:33
mentioned in commit 4d49d8c0a2ad3e94f44080bfe55cc853dd22198e

In GitLab by [Gitlab user @valentindavid] on Aug 9, 2018, 15:18
mentioned in commit 298fa0483ec407e0cc54c216eefcbeaf328d7f0a

In GitLab by [Gitlab user @valentindavid] on Aug 10, 2018, 09:53
mentioned in commit c5362746c10efc734fe65d5978d8c1ad47b3f0d9

In GitLab by [Gitlab user @valentindavid] on Aug 10, 2018, 11:08
mentioned in commit 9daa8eace89f750a227e38b59137e6861c256dd1

In GitLab by [Gitlab user @valentindavid] on Aug 10, 2018, 14:50
mentioned in commit 8757117c2fe3abd89e9682909587657125e6a10d

In GitLab by [Gitlab user @valentindavid] on Aug 10, 2018, 15:22
mentioned in commit 1b9c9277e92f57d4ec2bf510eaf00d45bf33ee12

In GitLab by [Gitlab user @valentindavid] on Aug 12, 2018, 06:54
mentioned in commit 322cab58e255916da10e0a59ae91e92f50898504

In GitLab by [Gitlab user @valentindavid] on Aug 12, 2018, 06:57
mentioned in commit 465909ddc6d39822f7669d554f3033e497b6f762

In GitLab by [Gitlab user @tristanvb] on Aug 12, 2018, 07:59
closed via merge request !616

In GitLab by [Gitlab user @valentindavid] on Aug 12, 2018, 07:59
closed via commit 322cab58e255916da10e0a59ae91e92f50898504

In GitLab by [Gitlab user @tristanvb] on Aug 12, 2018, 07:59
mentioned in commit c1fdebb3137891305bea7db8b4f1ad9f0b1e2043


Fix Issue887

Fix issue #887

Thanks - I'd merge this if you can resubmit it as a clean PR. Not 6 commits, and not with a bunch of formatting changes to other files which make the diff unreadable. Also, please make sure the issue title reflects the change vs just saying the bug number -- who remembers those?


Create a GitHub  templates

Thanks for this PR, @Devinsuit! Few comments, otherwise these look good.
One side note: If you include the related issue in the Git commit message (e.g. Closes #872), GitHub will automatically note that this PR relates to the Issue, and will also automatically close that issue when we merge this PR. It's quite nice! More on that: https://help.github.com/articles/closing-issues-via-commit-messages/

Merged as f49abcfc6ffb99740fe23fb48bad8f236ee2884a. Thanks, @Devinsuit!