Question regarding IFillableArmor and the Gas Charger

I've been working on an armor mod that add's an environmental suit (think Prey/Half-Life) and was looking at the interfaces AR provides. Most of it seems pretty straight forward though I do have a question about IFillableArmor. It looks like the Gas Charger/Oxygen Charge specifically looks for ItemSpaceArmor rather than an item that implements IFillableArmor. That's not really a problem if you extend the AR Class, but in my instance that's not really possible. I'm curious if that's an intended function or just a bug that needs to be fixed. It'd definitely be handy if checked for IFillableArmor instead.
https://github.com/zmaster587/AdvancedRocketry/blob/61d7b2b0bd9ee58a5cc67ab6c2bbb3b00328e0d6/src/main/java/zmaster587/advancedRocketry/tile/oxygen/TileOxygenCharger.java#L65-L72

IFillableArmor was added much later, this was just a reference that was missed, I'll get it corrected.


Finishing up mir-gen WIN64 ABI support

My initial va implementation assumed that va_list won't be passed to other functions. The first commit fixes that. The second commit adds multiple return values checks.

I've merged you pull request.  Thank you for working on Win64 ABI.


Link needed

It would be helpful if there was a link provided to NAVApp.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 4aad761f-ad97-7f15-fd0b-81f5716d2d90
Version Independent ID: d0f0fb7c-6a1f-efef-e43c-955bb10b1404
Content: Upgrading Extensions - Business Central
Content Source: dev-itpro/developer/devenv-upgrading-extensions.md
Service: dynamics365-business-central
GitHub Login: @jswymer
Microsoft Alias: solsen

Thanks for the feedback. This will be fixed with the next refresh of the docs.

Thanks for the feedback. This will be fixed with the next refresh of the docs.


Tweaks to run dspy-produced calls to the server, with gemma template.

Hi,
I am not sure if that is of general interest, but I thought I'd submit it in case it's useful.
I had to tweak the server a little to get it to interact with dspy following the following comment https://github.com/stanfordnlp/dspy/issues/385#issuecomment-1998939936.
The two tweaks are to (1) relax the validation and convert from int to float as needed, (2) to work around some chat template not taking the system role (such as Google Gemma's chat template).
can try it out with:
python -m server --model mlx-community/gemma-1.1-7b-it-4bit --port 1143

modulo patching the relative imports in server.py
-from .tokenizer_utils import TokenizerWrapper
-from .utils import generate_step, load
+from mlx_lm.tokenizer_utils import TokenizerWrapper
+from mlx_lm.utils import generate_step, load

and then, ont the dspy side:
import dspy
lm = dspy.OpenAI(model_type="chat", api_base="http://localhost:11434/v1/", api_key="not_needed", max_tokens=250)
lm("hello")

Thanks!

Cool! Btw you shouldn't need to change the imports if you run it as a package like so:
mlx_lm.server --model mlx-community/gemma-1.1-7b-it-4bit --port 1143

Done with your comments. Let me know if you prefer me to rebase. Thanks!

I tried a different application (OpenDevin), and to get it working with this server and gemma, I had to convert from OpenAI to Gemma messages as follows:
def openai2gemma(messages: List[dict]):
    print("before:", pretty_json(messages))
    for m in messages:
        m["role"] = m["role"].replace("system", "user")
    t_messages = []
    for m in messages:
        if t_messages == [] or t_messages[-1]["role"] != m["role"]:
            t_messages.append(m)
        else:
            t_messages[-1]["content"] = "\n" + m["content"]
    print("after:", pretty_json(t_messages))
    return t_messages

In short, replace system roles with user role, and make sure roles are strictly alternating.
Transforming from OpenAI messages to conform to a specific chat template is probably out of scope of this PR...

After further experimentation... the openai2gemma function seems useful for other models too, like llama3. In opendevin, mlx-community/Meta-Llama-3-70B-Instruct-4bit will silently work without the conversion function, but it produces questionable results.

@ awni a much better solution to deal with chat templates occurred to me. Just provide a --chat-template flag that is passed to the tokenizer. The provider is then responsible to adhere to the OpenAI chat model. For gemma, this template seems to work:
--chat-template "{{ bos_token }}{% set extra_system = '' %}{% for message in messages %}{% if (message['role'] == 'assistant') %}{% set role = 'model' %}{% else %}{% set role = message['role'] %}{% endif %}{% if role == 'system' %}{% set extra_system = extra_system + message['content'] %}{% else %}{% if role == 'user' and extra_system %}{% set message_system = 'System: ' + extra_system %}{% else %}{% set message_system = '' %}{% endif %}{{ '<start_of_turn>' + role + '\n' + message_system + message['content'] | trim + '<end_of_turn>\n' }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model\n'}}{% endif %}"

It basically does the convoluted logic in the jinja2 code.
What do you think?
Thanks!

I think that's great! Though I'm curious, where did you get the chat template for Gemma from? Is there a standard process for coming up with those?

Cool :)
I started with the original template, found like this:
% python
>>> from mlx_lm import load
>>>
>>> model, tokenizer = load("mlx-community/gemma-1.1-7b-it-4bit")
>>> tokenizer.chat_template
"{{ bos_token }}{% if messages[0]['role'] == 'system' %}{{ raise_exception('System role not supported') }}{% endif %}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if (message['role'] == 'assistant') %}{% set role = 'model' %}{% else %}{% set role = message['role'] %}{% endif %}{{ '<start_of_turn>' + role + '\n' + message['content'] | trim + '<end_of_turn>\n' }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model\n'}}{% endif %}"

and made it lenient instead of throwing exceptions when a system role is found or when roles do not alternate.
So in particular, like I was doing in the workaround in Python here earlier, I fold in the system messages to the next user message, prepending System:  and all the system contents accumulated in the meantime. That seems to work for most use cases. I don't bother checking alternation of roles at the moment.

I did some debugging on my gemma template, and I ran into the issue/misconception in jinja2 that variables outside a loop cannot be updated inside! I used a namespace as a workaround, as suggested here: https://stackoverflow.com/questions/46939756/setting-variable-in-jinja-for-loop-doesnt-persist-between-iterations.
So this is my updated chat-template parameter:
"{{ bos_token }}{% set ns = namespace(extra_system='') %}{% for message in messages %}{% set role = message['role'] %}{% if (message['role'] == 'assistant') %}{% set role = 'model' %}{% endif %}{% if (role == 'system') %}{% set ns.extra_system = ns.extra_system + message['content'] %}{% else %}{% set message_system = '' %}{% if (role == 'user') %}{% if (ns.extra_system == '') %}{% else %}{% set message_system = 'System: ' + ns.extra_system + '\\n' %}{% set ns.extra_system = '' %}{% endif %}{% endif %}{{ '<start_of_turn>' + role + '\\n' + message_system + message['content'] | trim + '<end_of_turn>\\n' }}{% endif %}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model\\n'}}{% endif %}"

For readability:
{{ bos_token }}
{% set ns = namespace(extra_system='') %}
{% for message in messages %}
  {% set role = message['role'] %}
  {% if (message['role'] == 'assistant') %}
    {% set role = 'model' %}
  {% endif %}
  {% if (role == 'system') %}
    {% set ns.extra_system = ns.extra_system + message['content'] %}
  {% else %}
    {% set message_system = '' %}
    {% if (role == 'user') %}
      {% if (ns.extra_system == '') %}
      {% else %}
        {% set message_system = 'System: ' + ns.extra_system + '\n' %}
        {% set ns.extra_system = '' %}
      {% endif %}
    {% endif %}
    {{ '<start_of_turn>' + role + '\n' + message_system + message['content'] | trim + '<end_of_turn>\n' }}
  {% endif %}
{% endfor %}
{% if add_generation_prompt %}{{'<start_of_turn>model\n'}}{% endif %}


Dockerfiles crashing when run

Description
Mxnet is not installed so the image crashes during training.
ModuleNotFoundError: No module named 'mxnet'

This happens for both Dockerfile and Dockerfile.gpu
Installing Mxnet dependencies into the dockerfiles should solve this.

@arangatang solved?


Bug: Bookmarklet removed in v0.7.2

Describe the bug
The bookmarklet link appears to have been commented out in 63c276a, which seems to be an unrelated commit.  As a result, it no longer shows on the "Add" page in v0.7.2.  It looks to me like this is unintentional, and I don't see any related issues.
ArchiveBox version
v0.7.2

Apologies, just found #1272 on another search.


Create positive at even and negative at odd in an array.c++

solved issue #202

Dude add this in correct folder


feat(build): upgrade dependencies

No behavior change, so no tests added. Tested by running go build, build succeeded.
$ go build .
go: downloading github.com/hashicorp/go-retryablehttp v0.7.7

Travis has run the unit tests.

:tada: This issue has been resolved in version 0.15.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Unread Chat badge that will never go away

Steps to Reproduce
My theory as to how this happened:

Receive DM from User A
Don't read the message
Account of User A is deleted, now I can't click the message to lower the unread count
Unread Chat badge for that message remains in the total count

Attachments

What platform(s) does this occur on?
iOS, Web (Desktop)
Device Info
No response
What version of the app are you using?
Build version: 1.92.1; Bundle info: 4e7b6bc (prod); Bundle date: 24103005; Platform: web
Additional Information
At this moment, my Chat icon has a (1) badge that will not go away.
I have triple checked and clicked on each chat thread in /messages on both iOS and Web to make sure it's not some other message.  I can't click on the deleted messages, so it must be the deleted users?

Update: I just noticed the tiny green dot in the upper-right of an unread message. When I executed the Leave Conversation menu item, the unread count was set to 0, and the badge went away. So, that is the work around.


Make it easier to click to select/deselect channels in ChannelConfigurationDialog

Due to the current implementation of selecting and deselecting channels, there is overlapping functionality between just click, and click-and-drag. If the mouse moves even the smallest distance while clicking it will generate a click-and-drag event which then has a rectangle that is too small to select anything, and also precludes the user from deselecting since click-and-drag is only used to select channels.
One easy step:

Check for the size of the selected rectangle, and if it is below a certain size then treat the event as a single click, which then gives back the power to deselect via a click event.

Other possible solutions:

Add in a button that needs to be held to deselect (i.e., Shift + click), which can be for Shift + click-and-drag as well to bulk deselect channels

Upon further testing, it appears that if there is more than one probe listed (such as in Rhs2116ChannelConfigurationDialog) the offset of the first probe is not taken into account when grabbing the index of the contact. This is an error, and should be corrected by applying the offset of all probes to the current index.

I did some integration testing and everything feels very nice. Not extensive though.


Add a link in each notebook that opens the notebook in GHG jupyterhub

Description
The documentation site comprises notebooks interacting with the US GHG Center data catalog/ services.
While these notebooks can be downloaded and run on a user's local computer, it's more convenient if they could be opened directly in the GHG Hub directly from the website.
Basically like what the data catalog page provides.
See here:
https://earth.gov/ghgcenter/data-catalog/eccodarwin-co2flux-monthgrid-v5
Click the access data button and click "Interactive Session in the US GHG Center JupyterHub (requires account)"
Example in veda-docs:
https://nasa-impact.github.io/veda-docs/notebooks/datasets/nceo-biomass-statistics.html
"Run this notebook" section (code here)
Acceptance Criteria

[ ] All the notebooks in ghgc-docs have a section with link to open the notebook in the GHG hub

nbgitpuller - https://nbgitpuller.readthedocs.io/en/latest/link.html


Migration to sequelize 3.0

Pull request for code with following changes:

migration to sequelize 3.0 from older unmaintained package sequelize-mysql
removed crypto module from package.json to use node core crypto module instead
removed space warning from signin.jade file
added IDE and vim temporary files to .gitignore

Sorry if things are a little bit unclean namely multiple commits in this pull request. I had to change stations midway and this is my first pull request ever on GitHub. I have tested all the code however and it all works correctly. I'll be happy to do any modifications given some directions.


feat(docs): add vhs tape for generating demo gif

Signed-off-by: Dolev Hadar<EMAIL_ADDRESS>

Looks good to me! 🚀


Issue #16 Fix Get-CacheEntry hash calculation

Update string Join to include the separator, so the GetHashCode is properly calculated based on the actual arguments.

Hi @chrissimon-au, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no Contribution License Agreement is required at this point. Real humans will now evaluate your PR.

TTYL, MSBOT;

See #16


2.0/Fix integration test - No conversion from FhirDateTime to Hl7.Fhir.Model.Quantity

See this page for most recent details
Library C# File: SevereObstetricComplicationsFHIR_0_1_000
Method : Risk_Variable_Heart_Rate_Value line 2055
Test Case ID = "6785c9c5-d58e-4e86-9f68-10c83597f8fb"
Observation ID = "delivery-date-estimated-124b"
Observation Value = "2025-01-31T08:00:00.000+00:00" (TypeName = dateTime)
Summary of Errors by Category
Unrecognized Guid format

AppropriateDXAScansForWomenUnder65FHIR

No conversion from  to Hl7.Fhir.Model.Quantity is defined.

AppropriateDXAScansForWomenUnder65FHIR

First BMI in Measurement Period: InvalidOperationException - No conversion from 9 to Hl7.Fhir.Model.Quantity is defined.
First BMI in Measurement Period Less Than or Equal to 20 kg m2: InvalidOperationException - No conversion from 9 to Hl7.Fhir.Model.Quantity is defined.
Has Risk Factor Active During the Measurement Period: InvalidOperationException - No conversion from 9 to Hl7.Fhir.Model.Quantity is defined.
Denominator Exclusions: InvalidOperationException - No conversion from 9 to Hl7.Fhir.Model.Quantity is defined.

UrinarySymptomScoreChangeAfterBenignProstaticHyperplasiaFHIR

Denominator Exclusions: InvalidOperationException - No conversion from 3 to Hl7.Fhir.Model.Quantity is defined.
Has BMI Exam Result Greater Than or Equal To 40 During Measurement Period and On or Before Follow Up USS Assessment: InvalidOperationException - No conversion from 3 to Hl7.Fhir.Model.Quantity is defined.
Has Morbid Obesity Diagnosis or BMI Exam Result  Greater Than or Equal to 40 Starts On or Before Follow Up USS Assessment: InvalidOperationException - No conversion from 3 to Hl7.Fhir.Model.Quantity is defined.

etc...

No conversion from  to Hl7.Fhir.Model.Quantity is defined.

SevereObstetricComplicationsFHIR

Risk Variable Heart Rate: InvalidOperationException - No conversion from 2025-01-31T08:00:00.000+00:00 to Hl7.Fhir.Model.Quantity is defined.
Risk Variable Heart Rate: InvalidOperationException - No conversion from 2025-02-01T08:00:00.000+00:00 to Hl7.Fhir.Model.Quantity is defined.
Risk Variable Heart Rate: InvalidOperationException - No conversion from 2025-02-05T08:15:00+00:00 to Hl7.Fhir.Model.Quantity is defined.
Risk Variable Heart Rate: InvalidOperationException - No conversion from 2025-01-31 to Hl7.Fhir.Model.Quantity is defined.

etc...

Cannot check equivalence for type KeyValuePair`2

HIVRetentionFHIR

Has Two Encounters With HIV At Least 90 Days Apart: ArgumentException - Cannot check equivalence for type KeyValuePair`2
Numerator: ArgumentException - Cannot check equivalence for type KeyValuePair`2

Slack reply from Rich Almeida
I think this is failing because there is no code filters on the observation, line 2016. These records shouldn't even be considered further in the code.
The filter in the elm, line 7242 looks like this:
"expression" : {
"locator" : "359:31-359:55",
"dataType" : "{http://hl7.org/fhir}Observation",
"templateId" : "http://hl7.org/fhir/StructureDefinition/heartrate",
"type" : "Retrieve",
"resultTypeSpecifier" : {
"type" : "ListTypeSpecifier",
"elementType" : {
"name" : "{http://hl7.org/fhir}Observation",
"type" : "NamedTypeSpecifier"
}
}
I knew at some point we might need to be using the data models, but I am not sure how that should be used or if there is enough info in the ELM to not worry about it. Do we attempt interpret templateId if it exists?
Here is the data model link for this https://github.com/cqframework/clinical_quality_language/blob/master/Src/java/quick/src/main/resources/org/hl7/fhir/qicore-modelinfo-4.1.1.xml#L6202
You can see that the name "observation-heartrate" should translate to
http://hl7.org/fhir/StructureDefinition/heartrate
and that would be the filter on observations
Then to retrieve value it provides the datatype.
https://github.com/cqframework/clinical_quality_language/blob/a022fc510b8ee868cc6d[…]uick/src/main/resources/org/hl7/fhir/qicore-modelinfo-4.1.1.xml
Here is a sample of a heart rate patient:
https://github.com/cqframework/ecqm-content-qicore-2024/blob/389b55e7b9a71c78c960f[…]CMS1028FHIR-v0.1.000-RISKPass-RISKHR24hoursB4EDOTTriageEnc.json (edited)
Rich Almeida
15 hours ago
This is not to say the measure is written correctly... I will try to run it other Java tooling, but its not a quick thing and last I checked there were some issues getting it off the ground.


Fix 404 error

broken links to python files on Index of Python Files is fixed

Hi Sandeep.  I just submitted PR 91 for same issue.  I suspect your solution is the desired one in case Mr. Norvig is looking at them both.  Mine simply forced direct to branch (in case he intended that, but doubt that)!

Greetings Mr. Petty. I  might have understood this issue superficially and I am not sure of my solution either. Let us count on Mr. Norvig for accepting the appropriate PR.


Fix AttributeError: module 'jax.random' has no attribute 'KeyArray' in Python 3.12

#1175

Resolved.


[Feature] Stop loading of page

In Chrome, there is an option to cancel loading of a page by clicking the X which is replaced by the refresh button when the page is loading.
There are some websites that keep on loading, even after 90s I keep on getting timeout errors.
If there was an option to stop loading the page (like there is in chrome), I would get the content that was already loaded and prevent from puppeteer to throw timeout.
I tried to used page.keyboard.press('Escape'); but with no luck..
Another solution would be to stop loading the page after X ms with something like that:
page.setPageLimitLoadingTime(30000);
which will stop the page from continuing the loading process and return all the data it already got...
Chromium API reference:
https://chromedevtools.github.io/devtools-protocol/tot/Page#method-stopLoading
Tell us about your environment:

Puppeteer version: 1.8
URLs (if applicable): https://bestmodelsbrasil.blogspot.co.il website with more the 90s loading time
Node.js version: 8

Thank you.
** if there already is an option for my proposal I'm sorry, I just couldn't find anything...

@RebliNk17 there's a window.stop method, would it be helpful to you?
await page.evaluate(() => window.stop());

Correct me if I'm wrong, but I think that the evaluate function is only running after page.goto finish...
anyway. it's not working.
something like this is partially working:
 await page._client.send("Page.stopLoading");
but I cannot find a way to tell puppeteer that the page has finished loading...

Don't know how and why this code now works:
await page._client.send("Page.stopLoading");
It stops loading the page and returns all the data...

Sorry, not working as I thought.
When using this flag: networkidle0 or networkidle2 in page.goto I am still getting timeout.
When using 'domcontentloaded' or 'load' I'm not getting all the data from some websites but than Im not getting timeout errors.
@aslushnikov Any thought on how to do it?
I've tried this:
https://github.com/RebliNk17/puppeteer/blob/master/lib/Page.js
But I'm still missing something...

@RebliNk17 what do you expect to see when you "stop" loading?
If you just want the navigation promise to not hang, I'd implement stopping somehow like this:
let stopCallback = null;
const stopPromise = new Promise(x => stopCallback = x);

const navigationPromise = Promise.race([
  page.goto(url).catch(e => void e),
  stopPromise
]);

// Do something; once you want to "stop" navigation, call `stopCallback`.
stopCallback();

@aslushnikov
I'll try to explain better my problem.
What I want is to receive the website HTML content and HTTP requests from the page.goto promise after X seconds passed without Exception even if the page did not finish loading.
Currently, if the page did not finish loading a Timeout exception is thrown and no data (HTML / HTTP requests etc) is returned.
Expected result:
When stopLoading is called, the page will stop all process (Just like when pressing ESC or the X button on a regular browser) and will "display" all the content that has been loaded until that press.
Is it clearer now?
If not, I will create a short video to explains it (English is not my native language)

Maybe page.goto(url, { waitUntil: 'domcontentloaded ' }) will suffice?

@aslushnikov Any thought on the code I shared above?
I work as expected but not sure if there is a better approach for that...
If it's good, should I create PR?

@RebliNk17 sorry for the delay, I was busy with other stuff.

Any thought on the code I shared above?

Can we step back and re-iterate since I still don't understand what's not working.
If I understand correctly, there's a website that takes a lot of time to load. We want to constrain wait time to certain amount and get content from the page after this time. Is this correct?
If yes, why's the following not working for you?
const puppeteer = require('puppeteer');
(async() => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  try {
    // Contrain loading time to 30 seconds
    await page.goto('https://bestmodelsbrasil.blogspot.co.il', {waitUntil: 'networkidle0', timeout: 30000});
  } catch (e) {
  }
  console.log(await page.content());
  await browser.close();
})();

Sorry, I did not get any notification about your comment.
Your code will work, but sometimes, the timeout might not be a time, it can also depended on different code running in the background, like in my situation.
Adding this "stopPageLoading" which exists in the Chromium API, will make it possible...
It's also something that Headless Chrome should have, even the regular chrome has it...

@aslushnikov Any thoughts?
I see two people voted this...

window.stop() is really work!!
Below is my typescript code sample:
page.goto('https://some site slow...');
await new Promise(resolve => setTimeout(resolve, 3000));
await page.evaluate(_ => window.stop());
await browser.close().catch(reason => console.error(reason));

There is actually no unhandled promise rejection from the goto invoke.

Your code will work, but sometimes, the timeout might not be a time, it can also depended on different code running in the background, like in my situation.

Wouldn't the solution in https://github.com/GoogleChrome/puppeteer/issues/3238#issuecomment-421762561 work in this case?

No, it wouldn't work...

Can you explain why?

Again, this is something that exists in the Chromium API... all that is needed it the implementation in puppeteer...

It's very important to understand what we add and why - otherwise we risk to bloat API with one-off solutions.

No, it wouldn't work...

Can you explain why?

Again, this is something that exists in the Chromium API... all that is needed it the implementation in puppeteer...

It's very important to understand what we add and why - otherwise we risk to bloat API with one-off solutions.

This code is not returning the page in case of the stopPromise wins the race...

This code is not returning the page in case of the stopPromise wins the race...

Well, that's very easy to address:
let stopCallback = null;
const stopPromise = new Promise(x => stopCallback = x);

const navigationPromise = Promise.race([
  page.goto(url).catch(e => void e),
  stopPromise,
]).then(() => page);

// Do something; once you want to "stop" navigation, call `stopCallback`.
stopCallback();

When trying to use page.status() getting page.status is not a function

@aslushnikov any update?

When trying to use page.status() getting page.status is not a function
with your code

@RebliNk17 There's no status method no a page object: Page API.
I guess what you want is a navigation response, not the page object.
In this case, the following should work:
let stopCallback = null;
const stopPromise = new Promise(x => stopCallback = x);

let navigationRequest = null;

const onRequest = r => {
  if (r.isNavigationRequest())
    navigationRequest = r;
};

page.on('request', onRequest);
const navigationPromise = Promise.race([
  page.goto(url).catch(e => void e),
  stopPromise,
]).then(() => {
  page.removeListener('request', onRequest);
  return navigationRequest.response();
});

// Do something; once you want to "stop" navigation, call `stopCallback`.
stopCallback();

Sorry, status of page respone. It is exists...
I am currently using it. I will check your code but i am not sure that this is what i am looking for.

@aslushnikov I am running into some difficulties here too.  I think part of the problem is just that there are a lot of different ways a page load can fail.
I'm building an archiving tool, and I'd like to give page.goto a chance to load a page for a while, and then stop the loading process and inspect what (if anything) is loaded so far.
If the page is partially or mostly loaded, but the browser gets stuck "Connecting..." to the server for a resource in the main rendering path, then page.goto will fail.  In this case (and potentially others) we can do better than the default timeout behavior:  we want to wait a while and let the browser try to load the page, then stop the loading process and see if we've got any usable content.
Currently I am getting the best results from the combination of the Promise.race technique above, and page._client.send( 'Page.stopLoading' ).  This works in more cases than window.stop().  For example, when the request for the main page is still "Connecting...", window.stop() won't work because no DOM has loaded).
One improvement in Puppeteer would be to add a Page.stopLoading() method, and/or make page.goto abort when a Page.stopLoading is sent to the same page, or when the page load is otherwise aborted (user clicks the Stop button, for example).  As things stand now, if you stop its page load elsewhere then page.goto will just time out, so some other code is needed to make this possible.

Same issue here, am in need of a "page.stopLoading()"

They don't care,
I gave them a sample, but a lot of things changed since then...
I can implement it again, but it's not very smart to adjust the code with every time a new version is out and I want to upgrade...

@aslushnikov I am running into some difficulties here too. I think part of the problem is just that there are a lot of different ways a page load can fail.

@nylen can you please help we understand what the problem is? Our previous discussion on the subject with @RebliNk17 has stalled.
You suggest adding a new method: page.stopLoading():

One improvement in Puppeteer would be to add a Page.stopLoading() method, and/or make page.goto abort when a Page.stopLoading is sent to the same page, or when the page load is otherwise aborted (user clicks the Stop button, for example). As things stand now, if you stop its page load elsewhere then page.goto will just time out, so some other code is needed to make this possible.

What would the method do? Why doesn't the workaround work for you?

@aslushnikov I think page.stopLoading() would just be a convenience method to do page._client.send('Page.stopLoading') behind the scenes.
The Promise.race workaround above is a decent starting point - it got me on the right track so thanks for that.  However, it is pretty clunky, and it doesn't actually stop the navigation. This means you could get more requests, a subsequent navigation redirect, etc which may be undesirable depending on the use case.
As said above, I'd also suggest that page.goto should be smart enough to detect (I suppose reject its promise with an appropriate error?) when navigation is stopped elsewhere, for example via 'Page.stopLoading' or maybe if a user clicks the browser's Stop button. This would allow simplifying the code to conditionally stop a page load even further, you'd just need to set a timeout that calls page.stopLoading and await page.goto(...) would still work as expected.

This means you could get more requests, a subsequent navigation redirect, etc which may be undesirable depending on the use case.

@nylen that's a good point. page.stop would prevent these, but page's javascript - if any - can still cause navigations and requests. Besides javascript, meta redirects might still cause navigations.
I think the biggest problem I have with page.stopLoading() is that it doesn't actually guarantee much.
It does abort all current in-flight requests - but I don't see why this can be useful.
Could it be that you rely on some specific behavior or page.stopLoading that I'm not aware of? Or maybe you can share your specific scenario so that I can feel your pain better?

My specific use case:  I was building a web archiving tool that (ideally) should work with arbitrary pages, and I found there are certain kinds of navigation timeouts that can be avoided or shortened, like when a page is stuck Connecting... to a resource that's in the main rendering path.  I think the issue in the OP is similar.
I agree there are other things that could cause navigations after a page is "stopped".  I am assuming that "aborting all current in-flight requests" is good enough for my use case, and so far it seems to be working.  For this part page._client.send('Page.stopLoading') is fine, but it was a bit hard to track down the correct call.  At least now that is documented in this issue.
So I am mostly just looking for potential ways to improve the code of puppeteer users here.  Hence the suggestion to make page.goto aware of "navigation aborted" events, because I think this would allow getting rid of the Promise.race in the examples above.
I don't think any of this is particularly urgent.  Thanks for all of your work on Puppeteer.

@aslushnikov Thank you for this

@RebliNk17 sorry for the delay, I was busy with other stuff.

Any thought on the code I shared above?

Can we step back and re-iterate since I still don't understand what's not working.
If I understand correctly, there's a website that takes a lot of time to load. We want to constrain wait time to certain amount and get content from the page after this time. Is this correct?
If yes, why's the following not working for you?
const puppeteer = require('puppeteer');
(async() => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  try {
    // Contrain loading time to 30 seconds
    await page.goto('https://bestmodelsbrasil.blogspot.co.il', {waitUntil: 'networkidle0', timeout: 30000});
  } catch (e) {
  }
  console.log(await page.content());
  await browser.close();
})();

I am using pyppeteer. and had the same problem. This solved my problem. i can access the DOM with.
page.content()

and session by
page.session()

I dont understand what everyone else is complaining about. Thank you soo much. Saved me couple;e of hours.

@RebliNk17 what do you expect to see when you "stop" loading?
If you just want the navigation promise to not hang, I'd implement stopping somehow like this:
let stopCallback = null;
const stopPromise = new Promise(x => stopCallback = x);

const navigationPromise = Promise.race([
  page.goto(url).catch(e => void e),
  stopPromise
]);

// Do something; once you want to "stop" navigation, call `stopCallback`.
stopCallback();

I tried your solution and it is working good but when i try to take screenshot i'm getting error
error: Error: Protocol error (Page.captureScreenshot): Unable to capture screenshot
    at Promise (/node_modules/puppeteer/lib/Connection.js:183:56)
    at new Promise (<anonymous>)
    at CDPSession.send (/node_modules/puppeteer/lib/Connection.js:182:12)
    at Page._screenshotTask (/node_modules/puppeteer/lib/Page.js:951:39)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  -- ASYNC --
    at Page.<anonymous> (/node_modules/puppeteer/lib/helper.js:111:15)
    at htmlBrowser (/dist/apps/botminds-browser/main.js:1079:45)
    at process._tickCallback (internal/process/next_tick.js:68:7)

Stop page loading and/or something else, this can also close the alert()
await page.keyboard.press('Escape')

If it doesn't work, then duplicate the line several times
await page.keyboard.press('Escape')
await page.keyboard.press('Escape')
await page.keyboard.press('Escape')

Can some one help me to stop this page of continuously loading ?
https://mbd.baidu.com/newspage/data/landingpage?s_type=news&dsp=wise&context={"nid"%3A"news_9644758218931914527"}&pageType=1&n_type=1&p_from=-1&rec_src=52
await page.evaluate(() => window.stop());
Not working, puppeteer just stuck.

This worked for me:
await page.goto(this.url, { waitUntil: 'domcontentloaded' })

Thanks!

@aslushnikov The problem is when setting a timeout with page.goto, even when it fails with TimeoutError, the page keeps running in the browser. This slows down the entire process. Sometimes browser.pages() takes 10+ seconds. Working in a Lambda environment leads to unpredictable behavior and various errors.
Whenever time is out and we reach the timeout, it would be great or even awesome to have a way to stop the page immediately. I agree stopping the ongoing requests won't helo much most likely but that'd be better than nothing.
Here's an example:
Function Logs
START RequestId: 44a07ce9-8800-4d98-bc5f-fdb236a34202 Version: $LATEST
2023-11-18T19:03:59.438Z	44a07ce9-8800-4d98-bc5f-fdb236a34202	INFO	Launching the browser
2023-11-18T19:04:01.978Z	44a07ce9-8800-4d98-bc5f-fdb236a34202	INFO	Creating new incognito context
2023-11-18T19:04:01.980Z	44a07ce9-8800-4d98-bc5f-fdb236a34202	INFO	Opening new page
2023-11-18T19:04:02.033Z	44a07ce9-8800-4d98-bc5f-fdb236a34202	INFO	Loading page
2023-11-18T19:04:05.452Z	44a07ce9-8800-4d98-bc5f-fdb236a34202	INFO	Main frame navigated to:  ...
2023-11-18T19:04:27.043Z	44a07ce9-8800-4d98-bc5f-fdb236a34202	INFO	Timeout error, skipping the page
2023-11-18T19:04:27.043Z	44a07ce9-8800-4d98-bc5f-fdb236a34202	INFO	Cleaning up
2023-11-18T19:04:27.050Z	44a07ce9-8800-4d98-bc5f-fdb236a34202	INFO	CLEANUP: Loading pages (await browser.pages())
2023-11-18T19:04:47.091Z	44a07ce9-8800-4d98-bc5f-fdb236a34202	INFO	CLEANUP: Closing pages (await Promise.all(pages.map(p => p.close().catch(() => {}))))
2023-11-18T19:04:47.111Z	44a07ce9-8800-4d98-bc5f-fdb236a34202	INFO	CLEANUP: disabling disconnect event handler (browser.off('disconnected'))
2023-11-18T19:04:47.111Z	44a07ce9-8800-4d98-bc5f-fdb236a34202	INFO	CLEANUP: Disconnecting from the browser (await browser.disconnect())
2023-11-18T19:04:47.112Z	44a07ce9-8800-4d98-bc5f-fdb236a34202	INFO	CLEANUP: Done
2023-11-18T19:04:47.112Z	44a07ce9-8800-4d98-bc5f-fdb236a34202	INFO	Gzipping
2023-11-18T19:04:47.113Z	44a07ce9-8800-4d98-bc5f-fdb236a34202	INFO	Responding
END RequestId: 44a07ce9-8800-4d98-bc5f-fdb236a34202
REPORT RequestId: 44a07ce9-8800-4d98-bc5f-fdb236a34202	Duration: 47680.63 ms	Billed Duration: 47681 ms	Memory Size: 1536 MB	Max Memory Used: 662 MB	Init Duration: 972.51 ms

You can see await browser.pages() took 20 seconds. What's worst, with lambda the page can keep running in the browser even after the function is restarted. So the next event starts opening a new page and then that context.newPage() takes an enormous amount of time.

Our product crawls our customer's website as part of our overall solution.  We are using Puppeteer for this and, overall, it works great.  But we have the same problem discussed here.  We can't know a priori what the appropriate timeout behavior needs to be on any given page or site.
When page.goto throws a TimeoutError, it doesn't necessarily mean that the page is unusable -- but after catching the error we can't access the HttpResponse that is returned when there is no exception.  If a new method, page.response(), for example, returned the response object if it is available, we'd be happy.  I realize that in some timeout scenarios the response will not be available (such as if the timeout is at the network layer).  It may also be a good idea for Puppeteer to emulate a "stop" when it throws an error, but I don't see that I need to be part of that.
So something like the following would be desireable:
const browser = await launch();
const page = await browser.newPage();
try {
  await page.goto(url, {waitUntil: ['load', 'networkidle2'], timeout: 30000});
catch(err) {
  // perhaps check for errors other than timeout
}
const response = await page.response();
if (response) {
  // use various information from the page itself, but also from response, such as url, status, headers
}
// clean up

for some reason this works for me:
await page
          .goto(url, { waitUntil: "domcontentloaded", timeout: 3000 })
          .catch((e) => void e);

await new Promise((resolve) => setTimeout(resolve, 3000));
await page.evaluate((_) => window.stop());

I used .preventDefault() for this:
`const labelElement = await page.getByRole('link', { name: 'See The Lakes' });
await labelElement.evaluate((element) => {
  element.addEventListener('click', (event) => {
    event.preventDefault();
  });
});

await labelElement.click();`


Deprecation warning

Getting the following error

Error: {"message":"starting container with HostConfig was deprecated since v1.10 and removed in v1.12"}

I'm using Docker 1.12.1 and running the latest published version of this module.

Hey @knownasilya,
Did you find a work around for this?
Cheers.

Referenced here (in go-dockerclient): https://github.com/fsouza/go-dockerclient/issues/537
Fixed here (in go-dockerclient): https://github.com/fsouza/go-dockerclient/commit/67c30ae246a11510ed1c3b72145e2daf76e4d664

No, I haven't found a work around.

docker run api doesn't accept parameters any more.
They should be passed during docker create.
This happens here on line 155
Removing the sopts from there fixes the problem of containers, however you'd want this configuration be passed during creation instead, hence see :
https://github.com/intval/docker-run/commit/8de675e99c82916ba43cd843a3c54d16169bfd44

@intval your fork worked for me, currently using that until it gets merged back upstream. Thanks!


fixes issue #306 and others: boolean attributes alignment with HTML

note: this change is not backward compatible

if the attribute foo is set to true in the vnode.data.attrs, it will appear in the dom as <div foo>
if the attribute foo is set to false in the vnode.data.attrs, it will not appear in the dom as <div>
snabbdom does not provide any validation, nor a whitelist of boolean, that's a responsibility of the creator of the vnode.
falsy values will be considered as regular attributes, and the value will be casted to string, just like html.

if the attribute foo is set to "false" in the vnode.data.attrs, it will appear in the dom as <input readonly="false">, which implies that the input is readonly.

perf optimization (this change should improve the perf a little bit)

Fantastic! I noticed the exact same thing when I saw #306—that this privously discussed change would fix it.
The breaking change seems to be for people who currently use truthy/falsy values to toggle boolean attributes. In my opinion that is bad practice and I assume it's rare.


Increasing dind worker nodes limit

I'm trying to increase my worker nodes to 3 and currently there is 2.
Couldn't find any workaround through the wiki.

Hi, try setting it via export NUM_NODES=3 The config vars are described in config.sh

Thanks and Worked .


Delete item with relationships: configure defaults

This is a continuation of https://github.com/DSpace/DSpace/pull/2577 and should only be reviewed after merging https://github.com/DSpace/DSpace/pull/2577 in master because it contains all the code from https://github.com/DSpace/DSpace/pull/2577.
This includes support for the configuration part of https://github.com/DSpace/Rest7Contract/pull/78

@benbosman : This PR can now be rebased on latest master (to make it much smaller & easier to review/test) as the related PR #2577 was just merged.  Thanks!

I've replaced this with https://github.com/DSpace/DSpace/pull/2599


Enable Detekt

Signed-off-by: Rupal Mahajan<EMAIL_ADDRESS>Description
Enable Detekt with upgraded version 1.22.0-RC2.
Issues Resolved
https://github.com/opensearch-project/dashboards-reports/issues/520
Check List

[ ] New functionality includes testing.

[ ] All tests pass, including unit test, integration test and doctest

[ ] New functionality has been documented.

[ ] New functionality has javadoc added
[ ] New functionality has user manual doc added

[ ] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #522 (bc7269b) into main (5fced6c) will increase coverage by 2.39%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##               main     #522      +/-   ##
============================================
+ Coverage     74.46%   76.85%   +2.39%
============================================
  Files            94       30      -64
  Lines          4022     1923    -2099
  Branches        641      412     -229
============================================
- Hits           2995     1478    -1517
+ Misses          893      442     -451
+ Partials        134        3     -131

Flag
Coverage Δ

dashboards-reports
76.85% <ø> (ø)

reports-scheduler
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...h/reportsscheduler/index/ReportDefinitionsIndex.kt

...rch/reportsscheduler/index/ReportInstancesIndex.kt

...nsearch/reportsscheduler/model/ReportDefinition.kt

...search/reportsscheduler/security/SecurityAccess.kt

...search/reportsscheduler/settings/PluginSettings.kt

...portsscheduler/model/ReportDefinitionDetailsDoc.kt

...tsscheduler/action/UpdateReportDefinitionAction.kt

...in/org/opensearch/reportsscheduler/util/Helpers.kt

...earch/reportsscheduler/metrics/RollingCounter.java

...sscheduler/model/UpdateReportDefinitionResponse.kt

... and 54 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


chore(deps): upgrade runtime dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-runtime-dependencies-k8s-25-main" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 3365

Questions ?
Please refer to the Backport tool documentation


Fix samples per second calculation bug introduced by ga_steps

I forgot to update this when I added ga_steps 😬

@lucidrains @rom1504 I'm considering just redesigning this and removing --ga_steps in favor of --batch_size and --micro_batch_size which is easier to understand for these purposes as your "effective batch size" remains "--batch_size".

yeah sounds good, also deepspeed has support for micro_batch_size , you don't have to compute it yourself ;)

would be cool to finish this one


Merge changes from prod-oldui branch to master

This change is

Coverage increased (+3.2%) to 50.397% when pulling d873b9d1af6f7f065a440cda8e0687644a635209 on ropable:master into 514ad327ba8f54eb69b53e5d94cdcb343f8ccc99 on parksandwildlife:master.


Store and reuse generated PyVerilator sim objects

When using either node-by-node or stitched-IP rtlsim, FINN currently re-creates the PyVerilator sim objects on every single execution. This can be quite a lengthy process, since it involves calling verilator and compiling the resulting C++ code behind the scenes.
A better approach would be to cache the PyVerilator sim objects by linking them somehow to the ONNX graph, so that they can be generated once and reused multiple times. One possible implementation of this is to pickle the PyVerilator sim object into a file, and add a node attribute pointing to this file, which can then be un-pickled for the next rtlsim execution.

It turns out that Python pickles won't work because of the need to serialize an open ctypes DLL handle, but therein also lies a very simple solution.
The workhorse of PyVerilator is Verilator, which compiles the circuit to emulate into a shared library file, which is then interfaces through Python. The returned PyVerilator object's so_file = sim.lib._name property returns the path to the shared library file, and can be "reloaded" by simply calling PyVerilator(so_file). The so_file string itself is easy to store as an ONNX node attribute.

Resolved in 6ca42a6605d3065c7ca9e2265dcf6eb3f85f9bcb
Both for node-by-node and whole-network stitched IP rtlsim, we now store the generated Verilator library filename as rtlsim_so attribute or metadata_prop. This is reused to create a PyVerilator wrapper on the next execute call.


Support for an initial before_attribute in stream_generator

Adds a start_after parameter to the stream_generator function so that an initial value for before_attribute can be set.
Example use case: a script wants to process all new comments from a sub since the last time it's been run. It keeps the fullname of the last comment it saw on its previous run and wants to stream all comments starting from the one after that. It can do:
for comment in reddit.sub.stream.comments(start_after=last_seen_fullname):
    # Do things...

For CHANGES.rst, should I increment the version number? (And if so do I need to do that elsewhere as well?) Or should I put it in Unreleased?

Put it in unreleased under the **Added** header.

Requested changes have been made.

Merged in! 🎉Thank you!

Thanks for all your help!

By the way, question: what's the release schedule of PRAW like? I want to use this in a project of mine but not sure if to just wait or to hack around it in the meantime

There isn't really a release schedule. However, you can do pip install --upgrade https://github.com/praw-dev/praw/archive/master.zip to install the dev version.


[r2] Allow Large Uploads in Local Wrangler

What this PR solves / how to test
Fixes #5821.
Author has addressed the following

Tests

[ ] TODO (before merge)
[ ] Included
[x] Not necessary because: One Line change ok? Not entirely sure

E2E Tests CI Job required? (Use "e2e" label or ask maintainer to run separately)

[x] I don't know
[ ] Required / Maybe required
[ ] Not required because:

Changeset (Changeset guidelines)

[ ] TODO (before merge)
[x] Included
[ ] Not necessary because:

Public documentation

[ ] TODO (before merge)
[ ] Cloudflare docs PR(s):
[x] Not necessary because: Limits for the wrangler API aren't documented in the CLI docs atm.

Thanks for the PR!


SELinux policy broke with the move to bubble wrap

Building a Fedora IoT commit with the latest osbuild, leads to:
setfiles: /run/osbuild/tree/etc/selinux/targeted/contexts/files/file_contexts.bin: context system_u:object_r:usbguard_conf_t:s0 is invalid

# with audit log:
type=AVC msg=audit(1596967177.989:613): avc:  denied  { nnp_transition nosuid_transition } for  pid=48411 comm="org.osbuild.sel" scontext=unconfined_u:unconfined_r:osbuild_t:s0-s0:c0.c1023 tcontext=unconfined_u:unconfined_r:setfiles_mac_t:s0-s0:c0.c1023 tclass=process2 permissive=0

The underlying issue seems to be the switch from systemd-nspawn to bwrap (#389). The latter always sets the no-new-privs flag, which is an opt-in for the former and osbuild did not opt in. Said flag interferes with the transition and additional rules are needed via the "nnp_nosuid_transition policy capability", like:
allow osbuild_t install_t:process2 { nnp_transition nosuid_transition };
allow osbuild_t setfiles_mac_t:process2 { nnp_transition nosuid_transition };

NB, this obviously also breaks deploying a commit:
ostree remote delete c15e385b-9742-4a19-be1e-c941099dae76 --repo=/home/gicmo/Code/src/osbuild-iot/build/store/osbuild-sources-output-luqzq66y/org.osbuild.ostree/repo
error: Writing content object: fsetxattr: Invalid argument
ostree admin init-fs --modern /run/osbuild/tree --sysroot=/run/osbuild/tree
ostree pull-local /run/osbuild/sources/org.osbuild.ostree/repo 90bdfea9821f97c682e156f0a8d5cf33bb23e1d21fde0113c4cdc12a93635144 --repo=/run/osbuild/tree/ostree/repo
Fetching ostree commit 90bdfea9821f97c682e156f0a8d5cf33bb23e1d21fde0113c4cdc12a93635144
Traceback (most recent call last):
  File "/run/osbuild/lib/stages/org.osbuild.ostree", line 337, in <module>
    r = main(stage_args["tree"],
  File "/run/osbuild/lib/stages/org.osbuild.ostree", line 239, in main
    ostree("pull-local", source_repo, commit,
  File "/run/osbuild/lib/stages/org.osbuild.ostree", line 141, in ostree
    subprocess.run(["ostree"] + args,
  File "/usr/lib64/python3.8/subprocess.py", line 512, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ostree', 'pull-local', '/run/osbuild/sources/org.osbuild.ostree/repo', '90bdfea9821f97c682e156f0a8d5cf33bb23e1d21fde0113c4cdc12a93635144', '--repo=/run/osbuild/tree/ostree/repo']' returned non-zero exit status 1.

We also need the nosuid_transition because bwrap always sets this flag for bind-mounts.


me.js assumes that chromium cannot play mp4

On most Linux distros, you can enable mp4 support, yet this line will always cause Chromium to fall back to Flash (or fail entirely if Flash isn't installed) regardless of whether a normal <video type="video/mp4" src="xyz.mp4"> could play natively.
I've been working around this by forcing mejs.MediaFeatures.isChromium to false. This flag is only used in the place described above.

The same applies for audio and mp3. It shows a download file button and when downloading the mp3 file it plays just fine within the browser. Could this be fixed and is there currently a workaround?

@rixth Closing old issues. If you'd like to reopen this, let us know. Thanks!

This issue is still pertinent nowadays. Testing with mediaelement.js 2.23.4 from Chromium on a standard linux mint install (and thus with mp4 support), http://html5test.com/ will report H.264 support, however mediaelement.js still restricts media support for Chromium to webm, ogv, and ogg due to the following
// special case for Chromium to specify natively supported video codecs (i.e. WebM and Theora)
if (result.isVideo && mejs.MediaFeatures.isChromium) {
	htmlMediaElement.canPlayType = function(type) {
		return (type.match(/video\/(webm|ogv|ogg)/gi) !== null) ? 'maybe' : '';
	};
}

Was that workaround because Chromium would report « maybe » for mp4 when not supporting it? I would think there would be more Chromium users nowadays with mp3/mp4 support then the opposite, and it's unfortunate that using mediaelement.js can remove support for those clients when only a mp4 file is available (because flash fallback won't work in most, being supported on linux only with Chrome).

This issue has been fixed in 3.x-dev branch and will be released soon. Please test it and let me know if it solves your issue

Just tested 3.x-dev branch, and it indeed solves this issue for me. Thanks.


Upload file problem

i use post of http to upload file,but a little of request be saved.another most requset not be saved in pinpoint. why?

Hi @symark
I don't understand what's going on. Could you explain it a bit more?

If it's a post request, can it be logged?

@symark Yes.

but i wanna logged~~~~~~~

Hi @symark
It might have to do with the sampling rate. Please check this section in the wiki, which just got added.
You'll have to set profiler.sampling.rate=1 in your agent's pinpoint.config file to trace every request.

thank you ,guy


Add support for SQL statements longer than 64KB

This patch allows Firebird to set the statement length, so you can execute SQL statements longer than 64K in Firebird 3.0.

Seems correct length is determined inside prepareStatement when length is 0
https://github.com/FirebirdSQL/firebird/blob/B3_0_Release/src/dsql/dsql.cpp#L1320


🐛 Logic: Gas limit override tx limit

Issue description
In the logic module the max_gas on-chain parameter is used to limit the gas consumption allowed when executing the Ask query.
In the context of a transaction execution the gas consumption limit is directly expressed in the transaction, so when the logic ask query is used in a transaction (e.g. called by a smart contract) having this parameter overrides the limit set at the transaction level and allows this way unexpected behaviours such as transaction gas over consumption, or transaction failing prematurely for false out of gas errors.
This issue is consensus safe in the sense that it is a deterministic behaviour, but can lead to transaction fees not reflecting the real underlying execution cost.
Proposed solution
I propose to remove this parameter, keep the gas consumption tracking while letting the ask logic query be unlimited in terms of gas consumption by default. If a gas limit should be positioned it should be done when forging the sdk context before calling the query.
Regarding the node operator perspective the gas limit being restricted by the transaction is quite normal, but as an RPC provider you may want to be able to restrict such queries as well, this already existing flag allows to set a global gas limit to all REST/gRPC queries and can serve as a replacement solution.
@bdeneux @ccamel

@amimart Yes, I think you're right! We should leverage the existing and common gas management in the SDK to avoid any misunderstandings in usage.
For nodes that expose REST and gRPC, we should consider adding a recommendation in the documentation to use the global gas limit for all REST/gRPC queries. Without this, a simple query could potentially block the node. However, it is ultimately the responsibility of the node host.


Throw error if an entry is too long for the tar archive format used by Bee

Swarm-CLI: https://github.com/ethersphere/swarm-cli/issues/250
Bee: https://github.com/ethersphere/bee/issues/2542

Figure out what tar format Bee is using
Check in bee-js if the data to be uploaded contains any invalid entries for that specific tar spec
Throw human friendly error

Release v8.3.0 addresses this.


Using @JavaProxy decorator does not yield DEX class

From @matthewstrom on January 6, 2017 1:18
Please, provide the details below:
Did you verify this is a real problem by searching [Stack Overflow]
Yes
Tell us about the problem
Using the @JavaProxy TypeScript decorator does not yield DEX class. However, implementing a proxy via the extend() method does work.
It's worth noting that, as far as I can tell, the nativescript package does not contain an implementation for JavaProxy; it only includes a type declaration
Which platform(s) does your issue occur on?
Android
Please provide the following version numbers that your issue occurs with:

CLI: 2.4.2
Cross-platform modules: 2.4.4
Runtime(s): 2.4.1
Plugin(s): N/A

Please tell us how to recreate the issue in as much detail as possible.

Delete the platforms/ folder to ensure there will be no residual files.
Add the TypeScript snippet below to your NativeScript project.
Run tns build android.
Look in the platforms/android/build/nativescript-bindings for the file com.tns.MyActivity.dex. The file will not exist.
Clean the platforms/ folder again.
Add the JavaScript snippet (the non-transpiled snippet) below to your project.
Run tns build android.
Look in the platforms/android/build/nativescript-bindings for the file com.tns.MyActivity.dex. The file will exist.

Is there code involved? If so, please share the minimal amount of code needed to recreate the problem.
Using code provided in the NativeScript blog post for Static Binding Generator.
Including the following JavaScript code in the NativeScript project will yield a corresponding DEX class:
JavaScript
android.app.Activity.extend('com.tns.MyActivity', {
	onCreate: function(bundle) {
		this.super.onCreate(bundle);
	}
});

However, including the following TypeScript code or even the transpiled JS code does not yield a DEX class:
TypeScript
@JavaProxy("com.tns.MyActivity")
class MyActivity extends android.app.Activity {
	protected onCreate(bundle) {
		super.onCreate(bundle);
	}
}

Transpiled JavaScript
var MyActivity = (function (_super) {
    __extends(MyActivity, _super);
    function MyActivity() {
        return _super.apply(this, arguments) || this;
    }
    MyActivity.prototype.onCreate = function (bundle) {
        _super.prototype.onCreate.call(this, bundle);
    };
    return MyActivity;
}(android.app.Activity));
MyActivity = __decorate([
    JavaProxy("com.tns.MyActivity"),
    __metadata("design:paramtypes", [])
], MyActivity);

Copied from original issue: NativeScript/NativeScript#3422

Hey @matthewstrom!
By the looks of it, I assume that everything is setup properly, and the TypeScript dependency's version in your project is 2.1+. The recent versions of the TypeScript transpiler broke the Static Binding Generator, but a fix has since been found and implemented. More details in the issue https://github.com/NativeScript/android-runtime/issues/651.
Now you can either manually pin the TypeScript version to 2.0.10 (npm install<EMAIL_ADDRESS>--save-dev), or fetch the latest android runtime from the master branch where the fix is present (tns platform add android@next)

@Pip3r4o
Upgrading to android@next solved the problem. Thanks.
This is not the first time that TypeScript 2.1+ has caused inexplicable errors for me.


feature/GDEV-732

Description:
- Refactored Modal into its own file.
- Style of modal elements more closely aligned with wireframes.
- Email body text refactored into a single variable.
- Other minor UX overhauls.
- closes GDEV-708
- closes GDEV-692
- closes GDEV-735
- closes GDEV-736
- closes GDEV-751
- closes GDEV-741

Using #52 to merge this.


Add brew cask formula to install release

Could not find one.
It's important to increase project adoption on mac.
See https://formulae.brew.sh/api/cask/vimr.json for inspiration
Great project btw

Closing in favor of https://github.com/Kethku/neovide/issues/713


Breaking change interpreting lowdb types

TypeScript Version:  2.8.0-dev.20180307 (works as expected in 2.8.0-dev.20180302)
Code
type False = "0";
type True = "1";
type Bool = True | False;
type If<Cond extends Bool, Then, Else> = { 1: Then; 0: Else }[Cond];

type Obj<T> = { [k: string]: T };
type UnionHasKey<Union extends string, K extends string> = ({ [S in Union]: "1" } & Obj<"0">)[K];

type ReferenceProperty = "@@reference";
type HasReferenceProperty<T> = UnionHasKey<keyof T, ReferenceProperty>;
type NonNull<T> = T & {};
type GetReferenceParent<T> = NonNull<T[keyof T & ReferenceProperty]>;

type GetDefinedReferenceType<TValue, SourceParent = GetReferenceParent<TValue>> = SourceParent[keyof SourceParent & ReferenceProperty];
type GetReferenceTypeIfDefined<T> = If<HasReferenceProperty<T>, GetDefinedReferenceType<T>, T>;
type SourceReference<T> = { readonly "@@reference"?: { readonly "@@reference": T } };

type T = GetReferenceTypeIfDefined<number & SourceReference<number>>; // Should be `number`
const x: T = ""; // Should fail

Expected behavior:
Error -- T is number
Actual behavior:
No error.
The example is based on lowdb on DefinitelyTyped.

Worked with @sandersn to get a simpler repro:
type GetAt<T> = T[keyof T & "@@@"] & {};
type GetAtAt<U> = GetAt<GetAt<U>>;
type V = GetAtAt<{ readonly "@@@"?: { readonly "@@@"?: number } }>; // Should be `number`
const x: V = ""; // Should fail

There is an error in 2.7 but not typescript@next. Removing the first ? in type V makes it correctly have an error in both versions.

From our offline discussion: this may be by design because T[keyof T & "@@@"] behaves like the conditional T extends { "@@@" } ? T["@@@"] : any.
? introduces undefined into the type, so the type of the object literal is
{ "@@@": { "@@@": number | undefined } | undefined }

That means that
GetAt<{ "@@@": { "@@@": number | undefined } | undefined }>

is { "@@@": number | undefined } | undefined. The second call to GetAt then looks like this:
GetAt<{ "@@@": number | undefined} | undefined>

And of course undefined doesn't have any keys (particularly, "@@@"), so keyof T is never, keyof T & "@@@" is never, and T[never] is any.


RDF Support Roadmap

This is an in progress issue that describes our roadmap for implementing Resource Description Framework (RDF) support in Kùzu. In Kùzu, users model their records as a set of node and relationship tables. So we make an explicit distinction between tables that store nodes and relationships and the storage/indexing of their properties/columns are different. We can call this the "structured property graph model". Ultimately this model is a simple variant of the standard relational model. RDF on the other hand is a separate graph-based data model. RDF is specifically suitable for modeling heterogenous, complex knowledge graphs. It has several advantages that don't exist in the relational model. Two of the important differences are (though there are others):

(i) Ability to define type/class hierarchies: In the relational model, the relation names can be interpreted as giving types of entities that are represented in your records but one cannot natively define a class hierarchy when modeling your records as one would do in object-oriented programming languages (e.g., that Cow is a Mammal and Mammals are Animals, etc.). RDF has a  standard vocabulary, rdf:type to define simple type/class hierarchies.

(ii) Ability to express schema and data in a homogeneous way: In the relational model there is a clear distinction between schemas, which are declared in CREATE Table Mammal(prop1, prop2, ...) statements, vs the actual data, i.e., the records that are inserted into these tables. One can query only the data records with explicit references to the tables/schema of the data records. In RDF, both schema and data are represented in the same records, which are "triples" that represent facts (explained momentarily). This is quite powerful when the modeled application domain is very complex and very hard to tabulate in a set of tables.

Overview of RDF
Subject, predicate, object triples and IRIs: In RDF, one represents a set of facts as (subject, predicate, object) triples. Subjects and predicates are always Resources (the "R" of RDF) that have globally unique identifiers called IRIs (often long strings that look like URLs).[^1] Two examples that indicate two facts about Justin Trudeau in the DBPedia RDF datasets are:

<https://dbpedia.org/page/Justin_Trudeau, https://dbpedia.org/ontology/almaMater, https://dbpedia.org/page/McGill_University>
<https://dbpedia.org/page/Justin_Trudeau, https://dbpedia.org/ontology/birthDate, 1971-12-25 (xsd:date)>

Objects in triples on the other hand can either be: (i) Resources; or (ii) Values, such as integers, strings, dates. Let's refer to these as "Resource objects" vs "Value objects". Triples can naturally be mapped to a graph abstraction as a set of edges. In the RDF community and the SPARQL query language that has been developed to query RDF databases, each (s, p, o) triple is thought of as an edge with label p s-[p]->o, where s is a Resource and o is either a  Resource or a value. Hence, RDF databases, i.e., sets of triples, are referred to as knowledge graphs.
Other components of RDF are as follows:
Blank Nodes: Some Resources may lack an IRI yet one may still indicate facts about such Resources. As an example, in the N3 raw file format to store triples, blank nodes can be expressed as follows: <dbr:Justin_Trudeau, dbo:spouse, [<dbo:birthDate, 1975-04-24 (xsd:date)>]> (I might be getting the syntax a bit wrong). The part between [ ] indicate a blank node (representing Sophie Trudeau) who has a dbo:birthDate property with value 1975-04-24 (xsd:date).
Named Graphs: An extension of RDF is to label each triple with the label of a graph the triple belongs to. These are called named graphs. For example if we have had an RDF named graph http://dbpedia.org/ that contained the triple about the alma mater of Justin Trudeau, we can represent it as a quadruple: <dbr:Justin_Trudeau, dbo:almaMater, dbr:McGill_University, http://dbpedia.org/>, where the last part is the IRI of the named graph. In SPARQL you can query over multiple named graphs and even attach variables to the named graphs.
Inference/Reasoning: The real power of RDF and the standards that come around RDF, such as RDFS and OWL, are that they provide a standard vocabulary around which one can develop a logical reasoner. This is quite advanced for us and beyond the scope of our initial roadmap (though arguably this is what makes RDF very powerful and super interesting) so I will also avoid getting into details but I will give a simple example. In RDFS (for RDF Schema), one can define the domains and ranges of predicates. You can say <dbo:almaMater, rdfs:range, dbo:EducationalInstitution> to specify the constraint that the range of dbo:almaMater is dbo:EducationalInstitution. From the triple about Justin Trudeau's alma mater above, a DBMS that has RDFS inference capabilities can deduce that <dbr:McGill_University, rdf:type, dbo:EducationalInstitution> even though this triple may never have been inserted into the system. Therefore, if you asked a query to return all triples <?, rdf:type, dbo:EducationalInstitution>, you'd get  dbr:McGill_University. A lot more advanced reasoning/inferences can be done automatically in the standards around RDF, such as OWL. This is the real strength of storing your records and the semantics of your records in RDF, which is founded on a logical formalism which facilitates reasoning over these records.
Goals of RDF Support in Kùzu
RDF and Kùzu's data model have differences but RDF triples can still be ingested into Kùzu as a set of nodes, relationships, and possibly some node properties and queries in Cypher. Our immediate goals in RDF support in Kùzu is twofold:

Quick triple ingestion from standard raw file formats, such as n3, json-ld, turtle, rdf/xml: As a first step for this, we need to define how these triples get mapped to Nodes, Edges and properties. I will make a proposal here but we have some options here as I will describe below.
Fast querying of triples in Cypher: Once we can ingest these triples into Kùzu, we want to support querying these triples in Cypher, thus benefiting from our fast many-to-many join and recursive join capabilities and efficient storage structures. This should also allow linking the Resources that are modeled in RDF Graphs with other Node and Relationship tables that store structured information and query them seamlessly.

Outline for the rest of the document:
We should go in iterations and implement increasingly more advanced features. I will propose a rough design for the first 2 obvious steps and also mention more advanced features, such as RDF-Star and inferencing, which should be on our roadmap.

Basic RDF support with named graphs but only supporting "Resource objects".
Support for "value objects".

Step 1: Basic RDF Support
In this version we only support ingesting triples whose objects are Resources (so we do not ingest and query triples whose object is a value, e.g., <dbr:Sophie_Trudeau, dbo:birthDate, 1975-04-24 (xsd:date)>).
I will use the following example in this section (JT stands for Justin_Trudeau):
dbr:JT, dbo:spouse, [dbo:birthPlace, dbr:Quebec] .
dbr:JT, dbo:almaMater, dbr:McGill_University .
dbr:McGill_University, dbo:locatedIn, dbr:Quebec .
dbo:almaMater, rdf:type, rdf:Property .

There are 5 triples (note the 1 triple for the blank node representing Sophie Trudeau). The objects of all of these triples are Resources. There is 1 blank node and 9 IRIs identifying 9 Resources.
RDFGraphs and Triple Ingestion Commands:
Users need to first create an RDFGraph in their databases before they can insert triples into it. So far in Kùzu, users can create either a Node Table or a Rel Table. RDFGraphs would be the third logical storage abstraction supported in the system. Implicitly an RDFGraph represents a pair of Resource Node Table and a Triples Rel Table. However, RDFGraph is a new and separate abstraction that the system and users know about. That is, an RDFGraph is not just a wrapper around Resource Node Table and a Triples Rel Table (as will become clear).
From the user's perspective, ingesting triples into a named graph should be as simple as running these two commands:
CREATE RDFGraph DBPedia
COPY FROM dbpedia.ttl INTO DBPedia

Users can create multiple RDF graphs and give each one an explicit name. For example a user can create CREATE RDFGraph YAGO, so we should start off by supporting named graphs.
Mapping RDF Triples to Node/Rel Tables
Once these commands are executed, internally we should be mapping these triples into Kùzu's property graph model. For the basic RDF support, where objects of the ingested triples need to be Resources (and not values), there is a very natural way to do this (and I doubt there is a reasonable alternative), which is: internally, we create one node table and rel table with the following schemas:

Node Table DBPedia.Resources(ID SERIAL, iri STRING, PRIMARY KEY ID): We cannot make the iri primary key because blank nodes do not have IRIs. We define an ID property with data typeSERIAL, which is a new feature in Kùzu to indicate that the values of the column are dense integers from 0, ..., numNodes.  This table will contain 9 nodes one for each resource and blank node. For example it can look like this:

| ID  | IRI                   |
| --- | ----------------------|
| 0   | dbr:JT                |
| 1   | dbo:birthPlace        |
| 2   | dbo:spouse            |
| 3   | dbo:almaMater         |
| 4   | rdf:type              |
| 5   | null                  |
| 6   | rdf:Property          |
| 7   | dbr:McGill_University |
| 8   | dbr:Quebec            |
| 9   | dbo:locatedIn         |

Note that the Resource with ID 5 has null IRI and corresponds to the blank node representing Sophie Trudeau.

Rel Table DBPedia.Triples(FROM DBPedia.Resources, TO DBPedia.Resources, pIntIRI INT64): pIntIRI stands for "predicate integer IRI" and stores the integer ID of the IRI of the predicate  in the Resources table. Observe that every IRI appears as a node in the Resources table even if it never appears as a subject or an object of a triple.  This will create the following Rel Table. Note that we store Rel Tables in our Lists storage structure, which is a disk-based CSR but I'm showing the records as a regular table for simplicity.

|_relID | FROM  | pIntIRI | TO    |
|-------|-------|---------|-------|
| 0     | 0     | 2       | 5     |
| 1     | 0     | 3       | 7     |
| 2     | 3     | 4       | 6     |
| 3     | 5     | 1       | 8     |
| 4     | 7     | 9       | 8     |

Internal _relID is a system level identifier property we give to every relationship in the graph. Let's ignore the _relID property and look at the first tuple (0, 2, 5) as an example. This is is the  <dbr:JT, dbo:spouse, [dbo:birthPlace, dbr:Quebec]> triple if you follow the mapping of IRIs/blank nodes to Resource IDs in the DBPedia.Resources node table.
Note 1: I highly suggest that we sort the triple edges in Lists based on predicate value. This will be critical in doing quick search of predicates over large adjacency lists.
Note 2: A big challenge will be that some of the backwards adjacency lists will be very large. For example in DBPedia, there is a Resource that appears as the object of 16M many triples. So we would have 2 backward adjacency lists of size 16M (one storing the pIntIRI and one storing the TO properties of the DBPedia.Triples Rel table). We need optimizations to improve the scalability of such very large adjacency lists.
Hash Index to store IRI-to-Resource ID Mapping: Along with these 2 tables, we need to use our Hash Index to map IRIs to Resource IDs. We already have the capability to do this for mapping string primary keys to system-level node offsets. However, this would be a case where we use the HashIndex for a non-primary key column. We do not need the reverse map because it is already stored in the IRI column of the Resources Node table as a column.
As a summary: Internally we use 3 storage structures to store the triples: 1) Resources Node Table; 2) Triples Rel Table; 3) IRI HashIndex. Each RDFGraph creates its own Resources, Triples, and HashIndices.
Querying RDF Triples in Cypher
Users can then query triples in their RDFGraph using the ()-[]->() rel pattern in their MATCH or OPTIONAL MATCH clauses as follows:
MATCH (s1:DBPedia)-[p1:DBPedia]->(so:DBPedia)-[p2:DBPedia]->(o:DBPedia)
WHERE s1.iri = "dbr:JT"
RETURN *

Output:
| s1  | p1  | so   | p2     | o   |
|-------|---------|-------|
| {ID: 0, iri: "dbr:JT"}  |  {_relID: 0, iri: "dbo:spouse"} |  {ID: 5, iri: null} |  {_relID: 3, iri: "dbo:birthPlace"} | {ID: 8, iri: "dbr:Quebec"} |
| {ID: 0, iri: "dbr:JT"}  |  {_relID: 3, iri: "dbo:almaMater"} |  {ID: 7, iri: "dbr:McGill_University"} |  {_relID: 4, iri: "dbo:locatedIn"} | {ID: 8, iri: "dbr:Quebec"} |

The query asks for a 2-path query starting from the subject is dbr:JT over a single RDFGraph. We could also ask queries over multiple RDFGraphs, e.g.:
MATCH (s1:DBPedia)-[p1:DBPedia]->(o1:DBPedia), (s2:YAGO)-[p2:YAGO]->(o2:YAGO)
WHERE s1.iri = "dbr:JT" and s1.iri = s2.iri
RETURN *

Or with a pattern that assigns multiple labels to nodes/relationships: MATCH (s:DBPedia|YAGO)-[p:DBPedia|YAGO]->(o:DBPedia|YAGO) RETURN label(s), s, p, o. Here label(s) will return the name of the RDFGraph, similar to how users can bind a variable to the name of a named graph. At least at the face of it, supporting these query variants do not seem to require any changes to our query planner and executor.
Note1: This example already shows us that RDFGraph is not just a wrapper to map triples to node/rel tables. For example, the query processor of the system needs to be aware that what users want is not the pIntIRI property that we stored in the Triples rel table but the string IRI versions of those. So for example in the second tuple, for the triple matching (s1:DBPedia)-[p1:DBPedia]->(o:DBPedia) part of the query, we return ({ID: 0, iri: "dbr:JT"}, {_relID: 3, iri: "dbo:almaMater"}, {ID: 7, iri: "dbr:McGill_University"}) instead of (..., {_relID: 3, pIntIri: 2}, ...). So we convert the pIntIRI: 2, which is what we store in the first row of the Triples table, into iri: "dbo:almaMater". The query processor needs to be conscious of such RDF-specific query processing when evaluating queries that refer to the Triples and Resources tables.
Note 2: We can without any special treatment support querying of RDFGraphs and explicitly defined Node and Rel tables. This should just work seamlessly. For example, one can have an Employee node table and RelatedTo Rel Table from Employee nodes to DBPedia.Resources nodes. One can do this:
CREATE Rel Table RelatedTo(FROM Employee, TO DBPedia.Resources)
// Some CREATE Statements to insert records in to RelatedTo

MATCH (s:DBPedia)-[p:DBPedia]->(o:DBPedia)<-[p2:RelatedTo]-(a:Employee)
WHERE s1.iri = "dbr:JT"
RETURN a

This query asks for all employees that are related to some Resource that Justin Trudeau has a connection to in the DBPedia graph. We could have also asked:
MATCH (a:DBPedia.Resource)<-[p2:RelatedTo]-(a:Employee)
WHERE a.name = "Alice"
RETURN a

This query asks for all Resource the Employee Alice is relatedTo. Note that I used the DBPedia.Resources label in variable a instead of DBPedia. See my next node on this.
Note 3:  In the first query we used this rel pattern (s1:DBPedia)-[p1:DBPedia]->(so:DBPedia) instead of (s1:DBPedia.Resources)-[p1:DBPedia.Triples]->(so:DBPedia.Resources), which is more verbose.  However in the immediately above query we used MATCH (a:DBPedia.Resource)<-[p2:RelatedTo]-(a:Employee) as it seemed more natural to be explicit that we were querying the Resources in DBPedia and that is how we had defined the RelatedTo table. We need to fix a consistent syntax here but assuming it is not hard to infer, we might just be flexible here and allow both syntaxes whenever they make sense.
Note 4: We can also allow queries that only query the Resources nodes and not triples as follows:
MATCH (s:DBPedia.Resources)
RETURN *

For now, I suggest we support this even if this deviates from the RDF/SPARQL standards one can query triples only using triple patterns (e.g., in SPARQL you have to bind both subject, predicate, and objects as in SELECT * WHERE { ?s, ?p, ?o }). So we would deviate from this norm but it is also less code on our side to restrict these queries.
Implementation Challenges: I won't go into many details here, but the main changes are to: (i) Catalog; (ii) execution of the COPY statement to a parser for Turtle, N3, Json-LD files etc; (iii) Planner and join optimizer to ensure we bind correct types etc.
Updating RDF Triples in Cypher
We need to support CREATE and DELETE statements. There is a decision to make here about whether we force users to CREATE and DELETE Resources and Triples separately or only as triples. I would suggest that we force users to CREATE and DELETE as triples and also ingest data in COPY FROM statements  as triples. This is another part we might want to get feedback from users and RDF community.
Note: When deleting triples, we cannot remove nodes from the Resources table or the IRI Hash Index. We can only remove relationships from the Triples table. So the Resources table and IRI Hash Index for RDFGraphs can only be growing. We can only remove relationships from the Triples table. The reason for this is this: when a triple (s, p, o) is deleted, we can check if  s.IRI or p.IRI etc. appear as subject or an object quickly by checking whether their adjacency lists in the Triples table are empty. But we cannot easily check if they appear as a predicate. This is because we do not have an index to check whether an IRI is stored as a predicate in some triple.
Step 2: Value Objects
Our second step is to support storing and querying triples whose objects can also be values, such as <dbr:ST, dbo:birthDate, "1975-04-24"^xsd:date>. This requires major changes to the system.
Let's change our example a bit:
dbr:JT, dbo:spouse, [dbo:birthPlace, dbr:Quebec;
                     dbo:birthDate, "1975-04-24"^xsd:date] .
dbr:JT, dbo:almaMater, dbr:McGill_University .
dbr:McGill_University, dbo:locatedIn, dbr:Quebec .
dbo:almaMater, rdf:type, rdf:Property .

I added one new triple to the blank node with  [dbo:birthDate, "1975-04-24"^xsd:date] as predicate and object.
Mapping RDF Triples to Node/Rel Tables
I will discuss two options for storing/mapping these triples to our Node/Rel Tables. The second one is suggested by Gaurav and Xiyang and is worth considering but we would need to do some back of the envelope calculations about its storage costs. My position to go with the first storage design.
Option 1: We extend the Resources node table with a semiStructuredTriples column and store a blob property for each node to store these triples in the form of intPredicateIRI:dataType:value triples. Note that this is semi-structured as we define the type of the data along with the data. So we would have DBPedia.Resources(ID SERIAL, iri STRING, semiStructuredTriples BLOB, PRIMARY KEY ID).
| ID  | IRI                   | semiStructuredTriples |
| --- | ----------------------| ----------------------|
| ... |     ...               |       ...             |
| 4   | rdf:type              | null                  |
| 5   | null                  |  {dbo:birthDate, "1975-04-24"^xsd:date} |
| ... |     ...               |       ...             |

Above {dbo:birthDate, "1975-04-24"^xsd:date} would not be stored as string but in an efficient binary encoding of these values. For example, the date would be stored in 4 bytes, data type xsd:date would be stored in 1 byte, and dbo:birthDate in 8 bytes in the integer version of the IRI.[^2]
Option 2: For each possible data type of the objects we store, we can extend the Resources table with a column: e.g., DBPedia.Resources(ID SERIAL, iri STRING, int64Predicate List<INT64>, int64Values: List<INT64>, floatPredicate List<INT64>, floatValues: List<FLOAT>,  ...., PRIMARY KEY ID). This may actually not too bad in storage given most things will be null and we will compress nulls very well with our Storage V2 design. It might be worth investigating as a side project. The real differences can be that this type of storage can slow down queries that need to check if a particular node has a particular predicate. That is because dozens of columns would need to be scanned as we are destined to support dozens of data types.
Irrespective of which choice we take, we also need to store the IRIs of the predicates in these triples in the IRI Hash Index and allocate them an offset in the Resources table. For example for the triple [{dbo:birthDate, "1975-04-24"^xsd:date}] of blank node, the predicate is dbo:birthDate and still has an IRI, so is a Resource and should be stored and mapped to an integer in the IRI Hash Index.
Querying RDF Triples in Cypher
Irrespective of how we store these triples, when querying these triples we have a choice. I'm a bit divided about this, but Option 1 below follows SPARQL more closely, so that is my preference for now. We should try to get some insights on this from outside.

We can interpret them as objects of triples similar to triples whose objects are Resources. Queries and outputs would look as follows:

MATCH (s1:DBPedia)-[p1:DBPedia]->(so:DBPedia)-[p2:DBPedia]->(o:DBPedia)
WHERE s1.iri IS NULL
RETURN *

Output (scroll to the very right to see the last column value of the last output tuple):
| s1  | p1  | so   | p2     | o   |
|-------|---------|-------|
| {ID: 0, iri: "dbr:JT"}  |  {_relID: 0, iri: "dbo:spouse"} |  {ID: 5, iri: null} |  {_relID: 3, iri: "dbo:birthPlace"} | {ID: 8, iri: "dbr:Quebec"} |
| {ID: 0, iri: "dbr:JT"}  |  {_relID: 3, iri: "dbo:almaMater"} |  {ID: 7, iri: "dbr:McGill_University"} |  {_relID: 4, iri: "dbo:locatedIn"} | {ID: 8, iri: "dbr:Quebec"} |
| {ID: 0, iri: "dbr:JT"}  |  {_relID: 0, iri: "dbo:spouse"} |  {ID: 5, iri: null} |  {_relID: 3, iri: "dbo:birthDate"} | **{value: "1975-04-24"}** |

Note: The variable o's type is sometimes a Resource and sometimes another type. We already have similar capabilities for returning variables that can bind to multiple Node or Rel Tables, e.g., the query MATCH (a) RETURN * would return nodes from all node tables in the database bound to the variable a. In this case, for RDF value objects, we should probably define a new type called Value or Literal that can store a wide range of types. Another option is to make any object always Value and extend Value also with a Resource type, so it can hold both a Resource or other values and return them in a single data type.

We can interpret them as properties of Resources. Queries and outputs would look as follows:

MATCH (s:DBPedia)-[p:DBPedia]->(o:DBPedia)
WHERE s.iri IS NULL
RETURN *

Output:
| s   | p   | o   |
|-------|---------|-------|
 {ID: 5, iri: null, dbo:birthDate:1975-04-24} |  {_relID: 3, iri: "dbo:birthPlace"} | {ID: 8, iri: "dbr:Quebec"} |

Note 1: TheWHERE s.iri IS NULL is a way to search blank nodes, though we can support a separate function such as bnode(s) to be more explicit.
Note 2: dbo:birthDate:1975-04-24 is now a property on s1. The obvious problem with this approach is that users no longer have a single way to query triples seamlessly as they do in SPARQL. To query if a Resource has a dbo:birthDate predicate, they have to query node properties, e.g., MATCH (s:DBPedia) WHERE s.dbo:birthDate = "1975-04-24" RETURN *, instead of writing a rel pattern.
Note 3: Our running example 2-hop query's output also changes if we model value objects as node properties. Recall that the query was: MATCH (s1:DBPedia)-[p1:DBPedia]->(so:DBPedia)-[p2:DBPedia]->(o:DBPedia). Now  that dbo:birthDate:1975-04-24 is not part of the Triples table, the system would not return the output tuple({ID: 0, iri: "dbr:JT"}, {_relID: 0, iri: "dbo:spouse"}, {ID: 5, iri: null}, {_relID: 3, iri: "dbo:birthDate"}, {value: "1975-04-24"}) because the last two variables {_relID: 3, iri: "dbo:birthDate"}, {value: "1975-04-24"} are now a property of so and not a triple. This is an important problem because that changes the meaning of the join. So for example, if you wanted to query all properties of a node with IRI dbo:xyz, you'd have to run 2 queries: One MATCH (s:DBPedia) WHERE s.iri = dbo:xyz RETURN *, and another one MATCH (s:DBPedia)-[p:DBPedia]->(o:DBPedia) WHERE s.iri = dbo:xyz RETURN *.
Implementation Challenge: Irrespective of how we model value objects, changing our query processor and expression and function evaluators will need to change a lot as our system can no longer be fully typed, i.e., there will be variables whose types we cannot infer during compilation time. So if we support running expressions and functions on those variables, we need a new unary and binary expression evaluators that assumes that the type of their operands may not be known apriori, so they first check their operands' types and then call a function (and maybe there is certain ways to cast Value's into specific classes for some queries).
Updating RDF Triples in Cypher
Assuming we pick Option 1, the CREATE and DELETE statements should be the same except we need new operators to be able to update the blobs that store the predicate value pairs as a node property efficiently.
Optimizations

Native Namespace Prefix Support: Catalog can keep the prefixes of the IRIs separately and we can encode IRIs as an integer followed by a string in the HashIndex. Or we can have a new type IRI that is an integer prefix followed by a string or we could store the prefix as an integer separately than the rest of the string. We may need to experiment with these different ways of storing.
FSST String Compression Support: We are keeping a global HashIndex that will serve as a string dictionary. However, the strings we will use for the Hash Index will be very long, so we would like to compress them. One problem is that this is likely to be a very large dictionary, i.e., it would be very difficult to update and/or change its codes. We can experiment with using a single FSST dictionary for all the IRIs. A more natural approach could be to have a trie index, instead of a Hash Index but that sounds like a lot of engineering. Trie index can naturally compress common prefixes but not sure how effective it would be.
Figuring from the rel patterns during compilation when a particular object maps only to Resources and not to values: Our primary performance problem will be to try to bind both a Resource and a Value to the same variables, e.g., o, that binds to objects. Therefore, whenever we can we should infer during compilation if o can only bind to Resources, so we can use faster query execution paths. For example, in this query MATCH (s:DBPedia)-[p:DBPedia]->(o:DBPedia)<-[p2:RelatedTo]-(a:Employee), o can only be a Resource as it has an incoming edge from an Employee node.

Advanced Features:
RDF-Star
I am intentionally leaving this blank to be designed later. RDF-Star is an extension to RDF to make statements about statements (as I understand also soon to be part of the RDF 1.2 standard). Here by a statement I refer to a triple as making a "statement" about something in the modeled application domain. So in RDF-Star, a subject or object resource can be an quoted triple. So <<dbr:JT, dbo:spouse, dbr:ST>, abc:reportedBy, abc:John_Doe> is a statement that John Doe  reported that JT's spouse is ST. Importantly <dbr:JT, dbo:spouse, dbr:ST> is not necessarily a triple in the database. So if you did WHERE ?s, ?p, ?o in My current thoughts are that support for mapping RDF-Star tup on defining a new resourceType property for Resources stored in Resources Node Table that can be: (i) Resource or (ii) Implicit_Statement.
Inference Features
It's also unclear what to do here. The DBMS community does not have a lot of expertise in inference algorithms and we need to understand this space much much better before we can propose a good design.
[^1]: Resources can be thought of as objects as in object-oriented programming languages and one can make statements about them.
[^2]: This is effectively a simpler implementation of the "Semi-structured Node Property Lists" we had before we made Kùzu open sourced (for any one interested: before November 2022, we had a feature to store arbitrary key value properties on nodes called "Semi-structured Node Property Lists", which we removed.

Semih –
This looks great. Today I'm busy preparing work so that I can take vacation next week. After I return I'll review this more in-depth and give any further feedback I may have. From a user perspective, this looks quite promising. I like the two main goals laid out.
There are a ton of RDF related technologies. Instead of trying to support them all directly (eg implementing your own SHACL validation), it might best to expose enough so that we can use existing external tools. I like the Kuzu strategy for supporting machine learning applications: be aware of popular use cases, don't reinvent the wheel, expose enough to plug into external tools. Also, many existing RDF tools are not fully compliant with standards. If common simple cases work well, I'm less concerned about weird edge cases.
Do you envision being able to export back out to RDF?
What happens when imported RDF is mixed with native prop graph data?
I'm personally OK with the lack of SPARQL support. In my group I'm trying to build support for moving away from Gremlin and SPARQL, and towards Cypher as our main graph query language.
One common technique when converting from RDF to a property graph is to somehow mark which nodes should become properties. OBO has created several tools for this and others are floating around. If adding anything other than the most basic RDF support, I might suggest something like this.
Namespaces are mentioned. This is something I'm curious about the the pure prop graph space. If we ever get a standard for property graph schema, I would love that that to be supported.
Tom

Hi Tom,
Yes, certainly we would support features to export out in different RDF formats. This should certainly be fine if the results contain triples stored in RDFGraphs but it's less clear what to do when the results mix triples stored in RDFGraphs as well as other Node/Rel Tables. We would either restrict exporting to cases when the query results contain only triples or adopt a set of exporting rules.
For Namespaces: Can you say more? I mentioned namespaces in the document in the context of an optimization to store the IRI strings more efficiently in the system. Beyond that we are not considering supporting a notion of "namespace" in the system but I may not know the features you have in mind.
Semih

Hi @semihsalihoglu-uw this looks really good -
FWIW, I did a quick implementation of RDF atop Kùzu by creating a custom RDFlib.Store plugin class where nodes and predicates get stored in Kùzu tables, much like you've described:
https://github.com/DerwenAI/kuzu-rdflib
This follows from work that our team at Derwen had been doing on kglab which is an open source PyData-ish abstraction layer to provide integration paths for different areas of graph technologies.
A benefit of using an RDFlib plugin is that so much of the W3C stack in Python comes along, nearly for free. In other words, this resolves the question of inference, validation, etc., since SPARQL, SHACL, OWL-RL, etc., become trivial to use.
Of course, requests to add or delete a single node or triple become more complex, although with a bit of juggling inside Cypher queries that's feasible.
In practice, something would need to manage the namespaces and prefixes, also as mentioned above. You might look at how we approached this in kglab — and I'm happy to talk through that and brainstorm approaches here.

https://github.com/DerwenAI/kglab
https://derwen.ai/docs/kgl/
https://derwen.ai/s/kcgh#88 (video)
https://www.linkedin.com/groups/6725785/


Add client example: terminal emulator for browser

Landed in https://github.com/metarhia/StarterKit/commit/ff00dd126ae9bfef1ad164a90cc296ab87c67b30, https://github.com/metarhia/StarterKit/commit/f00d091b97df1d4b58356313ac669a7fec85d989, https://github.com/metarhia/StarterKit/commit/8a04843bdfa948e56eb8ab904ebf6fec937c808c and https://github.com/metarhia/StarterKit/commit/d3684e550fda63c42da918d17a0d9e8845cb41b7


ngMocks.render and ngMocks.hide

Closes #291

v11.7.0 has been released and contains a fix for the issue. Feel free to reopen the issue or to submit a new one if you meet any problems.


Need help: Server does not render Voice or Video

Dear Team,
I hosted the tool in gcloud Compute Engine. it's running perfectly with no errors in console. each user can join the room, and it shows the other users also, however, does not show Video or Audio or these users.

The second user was sharing Video and Audio both in the room.
Below are my config file details.
server/config/config.js (adding only changed/impt parameters)
redisOptions : {},
listeningPort         : 443,
listeningRedirectPort : 80,
httpOnly              : false,
requireSignInToAccess : false,
    webRtcTransport :
    		{
        		listenIps :
	        	[
		        	{ ip: '<IP_ADDRESS>', announcedIp: null }
    			],
        		initialAvailableOutgoingBitrate : 1000000,
	        	minimumAvailableOutgoingBitrate : 600000,
    			maxIncomingBitrate              : 1500000
      		}

server/public/config/config.js
var config =
{
	loginEnabled     : true,
	developmentPort  : 3443,
	productionPort   : 443,
	multipartyServer : '<IP_ADDRESS>',
	turnServers      : [
		{
			urls : [
				'turn:turn.example.com:443?transport=tcp'
			],
			username   : 'example',
			credential : 'example'
		}
	],
 }

Change <IP_ADDRESS> in listen ip  to your exact private ipv4 and announceIP to your global ipv4.

i did. the new conf:
{ ip: '<IP_ADDRESS>', announcedIp: '<IP_ADDRESS>' }
but the same issue. Attaching a few error and info logs below, if you may help out.
  multiparty-meeting-server:ERROR:Room TypeError: Cannot read property 'user' of undefined   multiparty-meeting-server:ERROR:Room     at Room._handleSocketRequest (/home/sabyasachiba/multiparty-meeting/server/lib/Room.js:431:49)   multiparty-meeting-server:ERROR:Room     at Socket.peer.socket.on (/home/sabyasachiba/multiparty-meeting/server/lib/Room.js:349:9)   multiparty-meeting-server:ERROR:Room     at Socket.emit (events.js:198:13)   multiparty-meeting-server:ERROR:Room     at Socket.EventEmitter.emit (domain.js:448:20)   multiparty-meeting-server:ERROR:Room     at /home/sabyasachiba/multiparty-meeting/server/node_modules/socket.io/lib/socket.js:528:12   multiparty-meeting-server:ERROR:Room     at process._tickCallback (internal/process/next_tick.js:61:11) +718ms
errorLog_MM.txt

Check frirwall/NAT device that the media ports  40000 -49999 udp, are DNAT-ed correctly to your private IP.

 My official email address<EMAIL_ADDRESS>@misi may I connect you, if that's ok..

Sure of course.

The same to me, thank you so much.
but why function getListenIps sets announcedIp null ?
listenIP.push({ ip: iface.address, announcedIp: null });


Add a default port: 9042

If no port is explicitly configured for a session,
it will now use the default CQL port 9042 instead of returning
an error.
Tested with existing examples by passing various SCYLLA_URI values, both with and without specifying the port.
Pre-review checklist

[x] I have split my patch into logically separate commits.
[x] All commit messages clearly explain what they change and why.
[ ] I added relevant tests for new features and bug fixes.
[x] All commits compile, pass static checks and pass test.
[x] PR description sums up the changes and reasons why they should be introduced.
[x] I added appropriate Fixes: annotations to PR description.

In case you justly ask why I added collect()ing the iterator - that's because the return type of lookup_host is embedded in its type, so lookup_host<&str> returns a different iterator over SocketAddr than lookup_host<(&str, u16)>. I needed type erasure and collect() is a simple way of doing it.

Oh, one more thing - if we are using SSL, shouldn't we default to port 9142 instead of 9042? I think that's how it works in Scylla, but I'm not sure about Cassandra.

Cassandra generally uses the default port (source: https://docs.datastax.com/en/cassandra-oss/3.0/cassandra/configuration/secureCqlshSSL.html and similar). I figured that using a custom 9142 port would simply require specifying it explicitly. By the way, it's more complicated with Scylla, with certain configurations you get SSL on port 9042 if I remember correctly. 9142 is used if you want both non-SSL and SSL ports to be open, but that doesn't sound like a wise idea anyway.

I also grepped our java-driver, python-driver and cpp-driver, none of which mention 9142 anywhere, so it doesn't look like we have a pattern for providing a default SSL port.

OK, I see. Let's keep the port 9042 as the default in all cases, then.
Another thing that would be nice to do before merging is to add some documentation. Please mention the default port in the docstring of SessionConfig::known_nodes.

Right, done.

Thanks. Now it's perfect :)