ollama barely uses any Ram

Hey Guys,
I run ollama on docker and use mostly 7b models. But my Ram usage stays under 4 GB. Sometimes even below 3 GB.
But the recommendations are 8 GB of Ram.
It has 4 Core CPU, and it generates very slow even though I got 24 GB of Ram. I don't have a Video Card, though.
I'm new to this, so can anyone tell me what I might need to do differently?

Models are loaded using mmap and as a result probably appear in file cache memory use, rather than as part of the ollama process memory.

@neuleo for a 7b, 4bit quantized model I would expect it to take up around 4 GB. The amount of memory though comes down to the size of the model and the context size that you're using, so it's a bit squishy. We're adding some improvements in 0.1.19 to be able to more accurately guess the amount of memory though.
That said, I don't know what CPU you're using, but generally speaking, you'll get far better results from a GPU than the CPU. We've also got some changes coming to take more advantage of the AVX capabilities in the CPU which if you have a modern CPU w/ AVX-512 you may see some performance gains.
I'm going to go ahead and close the issue, but feel free to keep commenting or reach out on the discord.


initrd-0 hanging at 99%

Hello.
New to using Pixiecore, but seeing some odd behavior in my experiments and I thought you might be interested.  Hosting CoreOS images inside Pixiecore.  PXE booting a QEMU machine against Pixiecore and the kernel transfers ok but the initrd hangs at 99% the whole time.
My research suggests Pixiecore may not provide a content-length header for the images which is why iPXE skips to 99% .   Chances are my network config is just slow but I found the 99% part worth mentioning.
https://groups.google.com/d/msg/puppet-razor/wgzQzU8fzDY/BLUG4AdnZwsJ
Thoughts?
Thanks!

Hot damn, that's what was wrong! I've been beating my head against a wall trying to figure out why CoreOS seemed to take for ever to netboot.
I've built a hacky fix and CoreOS boot time on my pathological case laptop went from "never" to "seconds" with the addition of Content-Length in responses. I'll polish up the change and have it pushed within the hour.
Thanks!

Wow!  Fast turnaround.  Wow! Super fast downloading CoreOS now!.  Thanks!!


Home page image banner

Can we confirm that there will not be text here?
Can we confirm that it will not be interactive?
It probably will contain video: identify the video clips we like, and how they should go together?

superseded: this was based on previous design.


Show example on how to implement C function calls

Interop with C will probably be the primary way to get real world usefulness out of Cone, so that should be documented in my opinion.
How do I read a file?
How do I write to a socket?
How do I walk a directory tree?
etc.
So this is probably done by writing a wrapper around libc/muslc, right?
General usecases:

pass data or functions into C, get data or functions back
make those functions you pass in callable from either side

Yes, the documentation should include a page describing how to do interop with C.
By and large, it is pretty straightforward as Cone can map to C types directly. Function/type declarations need to be defined in Cone files, and will require extern to link them in from a library built using C.
I will write this up when I get the chance.

And here it is:  https://cone.jondgoodwin.com/coneref/refinclude.html#extern


feat(analytics-journeys): updated SDK to expose new analytics

Updated the SDK to provide exposure to the new api endpoint.

consumed by https://github.com/c-commerce/charles-browser-sdk/pull/335


Type on website

Hi,
Awesome plugin! There is a small typo on the website in the "Scrollable Sticky Element" section.
new stickySidebar  -> should be with capital.
Thanks,
Roy

I have fixed the sticky sidebar, and now it works perfectly.
Mocha Tests.pdf


Fix type check

Sorry for my late review. But https://github.com/launchableinc/cli/pull/492 was already merged within a day. I think this PR is no longer needed.


perf question

is it better to use single canvas context?
let canvasCtx: CanvasRenderingContext2D;

const getContext = () => {
  if (canvasCtx) {
    return canvasCtx;
  }

  return (canvasCtx = ...)
};

@AlanQtten we don't need to keep that in the memory. It's better to create a new one whenever needed.


Controller: added stats instrumentation

@ibuildthecloud @sheng-liang

there's no flush command support for the stats yet
instead the stats are flushed to controller stdout automatically upon the ping error

Can I also see all pending IO operations? You'll need to track all pending operations anyway in order to enforce timeout.

@kp6 I'd prefer we use https://github.com/rcrowley/go-metrics.  Also can we avoid locking.  There are points in the code where it's non-parallel like the beginning of the frontend or the rpc layer.

OK, but what would you like to do with that info? What's the use case?
-----Original Message-----
From: sheng-liang<EMAIL_ADDRESS>Received: Friday, 27 May 2016, 22:46
To: rancher/longhorn<EMAIL_ADDRESS>CC: Kirill Pavlov<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [rancher/longhorn] Controller: added stats instrumentation (#112)
Can I also see all pending IO operations? You'll need to track all pending operations anyway in order to enforce timeout.

You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/rancher/longhorn/pull/112#issuecomment-222291772, or mute the threadhttps://github.com/notifications/unsubscribe/AI_V78GKca-tT21Yi0dXIxlnq-xankdGks5qF9a3gaJpZM4IpAIs.

https://github.com/rcrowley/go-metrics is not an answer for tracking timeouts, that must be separate.  I'd probably just do that in that as a function of the RPC layer.

For troubleshooting, it is good to know not only what operations have recently completely, but also what are the on-going operations (which might be stuck, for example.)
From: Kirill<EMAIL_ADDRESS>Sent: Friday, May 27, 2016 11:07 PM
To: rancher/longhorn<EMAIL_ADDRESS>Cc: Sheng Liang<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [rancher/longhorn] Controller: added stats instrumentation (#112)
OK, but what would you like to do with that info? What's the use case?

@ibuildthecloud I looked the go-metrics but they don't seem to match what I thought we tried to achieve here:

we want a lightweight flight data recorder (FDR) that tracks stats and overwrites the old entries
the FDR logs nothing unless asked after the fact
the FDR is naturally customizable for whatever we need like pending I/O tracking

Re: the locking

the locks are taken very briefly but I'm sure it can be improved
I don't expect any noticeable contention or serialization effect there as the samples are typically at least 100us apart and putting the sample into the FDR takes less than 100ns (1000 times less)
could you point me to the non-parallel code you were referring to in the frontend and the rpc layer?
what's the concern there?

@sheng-liang I'll add the pending I/O tracking. I'll report current pending I/O along with the stats when they are flushed

Thanks. Tracking pending IO is probably required to implement read/write timeout anyway.
From: Kirill<EMAIL_ADDRESS>Sent: Saturday, May 28, 2016 12:01 AM
To: rancher/longhorn<EMAIL_ADDRESS>Cc: Sheng Liang<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [rancher/longhorn] Controller: added stats instrumentation (#112)
@sheng-lianghttps://github.com/sheng-liang I'll add the pending I/O tracking. I'll report current pending I/O along with the stats when they are flushed
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/rancher/longhorn/pull/112#issuecomment-222294134, or mute the threadhttps://github.com/notifications/unsubscribe/AJC73rzr94pq0lLt_MQh_DI1zZAKu9iNks5qF-hIgaJpZM4IpAIs.

@sheng-liang I added pending ops

@kp6 I guess I would say we don't need a FDR.  There's two objectives I've heard.  Handle operation timeouts and collect statistics.  For metrics, a metric library is obviously better.  For timeouts I'm not terribly sure what you have here is the best to handle that because this is out of band.  All you know is that some read/write timeout, but not the exact one.  We end up being limited on how intelligently we can deal with the timeout.

@kp6 I think we are going down the wrong path here.  The pending I/O commit now adds a separate data structure.  If the original sample code is not sufficient for pending I/O then should we delete it?  What does it accomplish.  The new code seems like an insert is on average O(n/2).  Now I'm sure you'd say that N is always small.
I think the hang up I have here is that I personally only care about one thing.  We want to move to per operation timeouts and not just rely on the ping.  I don't understand where this code is heading to see how we can effectively handle that.
Any code around metrics handling is a waste of time because this is well address subject in the community already.  All one needs to do is use a proper library like go-metrics and instrument your code to invoke gauges, meters, counters, timers, histographs, etc.

@ibuildthecloud the FDR is only for logging/reporting of recent and pending operations. It was not meant to serve any purpose in timeout handling except reporting upon timeout or on-demand when invoked via command line. The latter could be of form longhorn --report-ops or something like that. I do see your point on using something like go-metrics for the longer term. But not necessarily as a starting point. The use of go-metrics would imply somewhat different usage pattern:

a command line or configuration driven logging on/off control of the metrics regardless of timeout events or other errors
is that what you have in mind?

So how about this plan:

let me finish testing the non-ping based read/write timeout handling
let me address your other concerns:
I'll move the stats instrumentation into the rpc/client
I'll remove the redundant Sample API invocation as it is a subset of the new pending op tracking
O(n/2) bothers me too, and it's trivial to fix
then let's review the new PR and discuss switching to go-metrics as a separate issue?

would this plan work?

this PR was replaced by #118


Create a better looking comments

[ ] Create a slidedown, accordion-like comments section instead the current ugly popover.
[ ] Drop Bootstrap's popover.
[ ] Drop jQuery.

Hi @yammesicka, please assign to me

WIP:


[Feature Request] Integrate Openai and huggingFace model

Need to Implement openai and huggingFace model inside the models folder which we have.
openai model file -
https://github.com/promptslab/PromptifyJs/blob/f9176cb7b703995470f4095233bf6660f8839093/models/openai.js#L2
For hugging-Face -> We need to create a new file and then implement it
Node Package reference -

openai -https://www.npmjs.com/package/openai
huggingFace - https://www.npmjs.com/package/huggingface-api

Promptify reference -
hub-model - https://github.com/promptslab/Promptify/blob/main/promptify/models/nlp/hub_model.py
openai-model - https://github.com/promptslab/Promptify/blob/main/promptify/models/nlp/openai_model.py

openAI model integrated along with updated NER example.
Need to integrate - huggingFace model (Just to acknowledge the progress)

Hi @rk-teche   I’m the maintainer of LiteLLM (abstraction to call 100+ LLMs)- we allow you to create a proxy server to call 100+ LLMs, and I think it can solve your problem (I'd love your feedback if it does not)
Try it here: https://docs.litellm.ai/docs/proxy_server
https://github.com/BerriAI/litellm
Using LiteLLM Proxy Server
import openai
openai.api_base = "http://<IP_ADDRESS>:8000/" # proxy url
print(openai.ChatCompletion.create(model="test", messages=[{"role":"user", "content":"Hey!"}]))

Creating a proxy server
Ollama models
$ litellm --model ollama/llama2 --api_base http://localhost:11434

Hugging Face Models
$ export HUGGINGFACE_API_KEY=my-api-key #[OPTIONAL]
$ litellm --model claude-instant-1

Anthropic
$ export ANTHROPIC_API_KEY=my-api-key
$ litellm --model claude-instant-1

Palm
$ export PALM_API_KEY=my-palm-key
$ litellm --model palm/chat-bison


[type:refactor] refactor logging plugin

refactor logging plugin

Make sure that:

[ ] You have read the contribution guidelines.
[ ] You submit test cases (unit or integration tests) that back your changes.
[ ] Your local test passed ./mvnw clean install -Dmaven.javadoc.skip=true.

Codecov Report

Merging #3706 (e6bf2c9) into master (d02693a) will increase coverage by 0.33%.
The diff coverage is 68.88%.

@@             Coverage Diff              @@
##             master    #3706      +/-   ##
============================================
+ Coverage     63.04%   63.37%   +0.33%
- Complexity     5908     5944      +36
============================================
  Files           898      897       -1
  Lines         24727    24673      -54
  Branches       2253     2246       -7
============================================
+ Hits          15588    15637      +49
+ Misses         7729     7613     -116
- Partials       1410     1423      +13

Impacted Files
Coverage Δ

...u/plugin/logging/common/AbstractLoggingPlugin.java
0.00% <0.00%> (ø)

...in/logging/common/utils/LogCollectConfigUtils.java
73.80% <0.00%> (-12.31%)
:arrow_down:

...gin/logging/kafka/kafka/KafkaLogCollectClient.java
34.61% <20.00%> (-0.08%)
:arrow_down:

...ging/rocketmq/client/RocketMQLogCollectClient.java
56.60% <60.00%> (-5.40%)
:arrow_down:

...ugin/aliyun/sls/config/AliyunLogCollectConfig.java
97.67% <80.00%> (ø)

...g/kafka/handler/LoggingKafkaPluginDataHandler.java
72.00% <83.33%> (ø)

...enyu/plugin/aliyun/sls/LoggingAliYunSlsPlugin.java
100.00% <100.00%> (+9.09%)
:arrow_up:

...in/aliyun/sls/collector/AliyunSlsLogCollector.java
100.00% <100.00%> (ø)

...sls/handler/LoggingAliYunSlsPluginDataHandler.java
81.81% <100.00%> (ø)

...ging/elasticsearch/LoggingElasticSearchPlugin.java
100.00% <100.00%> (+9.09%)
:arrow_up:

... and 67 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d02693a...e6bf2c9. Read the comment docs.


Plugin install fails in docker if .meltano directory doesn't already exist

Migrated from GitLab: https://gitlab.com/meltano/meltano/-/issues/2498
Originally created by @cooley1 on 2021-01-05 20:09:05

What is the current bug behavior?
When running meltano install in a linux docker container, it will fail if the .meltano directory doesn't already exist. This means that you can't run meltano install on build if you don't have access to the db secrets during build.
What is the expected correct behavior?
If the .meltano directory doesn't already exits, it should be created.
Steps to reproduce
Run meltano install linux docker container that doesn't have a .meltano directory already built.
Relevant logs and/or screenshots
meltano.cli.utils.CliError: Can't initialize database: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/13/e3q8)
Can't initialize database: (sqlite3.OperationalError) unable to open database file
(Background on this error at: http://sqlalche.me/e/13/e3q8)

Possible fixes
On v1.62.0 it's here. We could add a mkdir command.
What's interesting is that on my Mac, sqlite will create the directory if it doesn't exit. But when I run it in the airflow image, it won't. Maybe it's a difference between permissions in Macos vs Linux? Not sure.
Further regression test
Ensure we automatically catch similar issues in the future

[ ] Write additional adequate test cases and submit test results
[ ] Test results should be reviewed by a person from the team

View 1 previous comment from the original issue on GitLab

Seems to be fixed now.


(default option to) disable all yellow topics-creation

Please add an option to not create any of the extra yellow topics during the PDF-import and processing workflow and enable it by default.
That includes the very source topic.
When I import the PDF "how to avoid big boats" under the "sailing"-concept of my knowledge tree, I don't need another yellow topic named "Sailing" below "Saililng" in which I can see the green PDF-topic "How to Avoid big Boats".
Reading "How to avoid big boats" and extracting the chapter "Various Types of big Boats", I don't want to see another yellow topic named "How to Avoid big Boats" under the green PDF-topic "How to Avoid big Boats".
When I wish to create an extract from the topic "Various Types of big Boats", I don't need another yellow topic named "Various Types of big Boats" between my extract and the text-extract "Containerships".
These extra yellow topics are a pain to manually remove and clutter up the Knowledge Tree for very little gain.
For those of use who use Neural Review and make Concept and Element-links, it's very important to be able to see the KT clearly without extra formatting noise.
Here's an example with "The Mind Illuminated" (as I'm sadly not an owner of "How to Avoid big Boats"
This:

looks much more difficult to parse at a glance than this:

I understand that this is to break up the number of elements accumulating within a branch so that noone would hit the default "children per branch" limit of 100. But for a PDF workflow, I believe it's reasonable to assume that almost no book has 100 chapters. Nor will any chapter yield 100 extracts. Nor is any extract likely to have 100 subextracts.
And for those few people who do run into this issue, it would be no trouble to simply set the limit to its maximum value of 500.

Not everyone extracts chapters into their own sub-PDF files, and most books will yield more than a 100 extracts.
Also most people have their children limit set to 100, and few of them know that this option even exist.
This will likely stay the default, but there could be an option in the future to disable the intermediate folder.

Not that opinionated about it having to be default or not. But I definitely do want that option.
An overnested KT looks like trash and I don't want it. I like pretty things.


PHI Settings

PHI permission enhancements

[ ] phi-access required property of permission
[ ] Container get/get_all methods use phi-access or phi flag to determine if a Request is valid and whether phi fields are accessible.
[ ] Do not log get_all when phi is not present

Custom PHI field settings

[ ] Allow users to set phi fields for each project if they are an admin and site level defaults if they are a site admin
[ ] Container post/put methods use phi-access or phi flag to determine if a Request is valid and whether phi fields are accessible.

Analysis POST requests

[ ] Use a union of the site level and project level fields when projecting phi fields

get_all should only use the site level

[ ] When post or putting a container, check that user isn't writing to any phi fields if they do not have phi-access
[ ] Add a way for Projects to override site level phi fields

If a project has phi=false, phi will always be returned from all endpoints and ignore site level phi settings.

If tags/notes are marked as PHI,
view? No
should a non-PHI user be able to add? - Yes
modify? - Yes for own notes, tags cannot be modified
delete? - Yes for own notes, no(?) for tags, but we'll allow for now.

Projects cannot override site-level PHI fields, the only way they can display fields blacklisted by site-level PHI settings is to list the project as phi=false.

What to do for /sessions when sessions will have different PHI settings? Use site-level defaults?

Log with PHI settings change in access log.


FIX: valid name logic

I introduced a bug in previous pull-request. The logic required an "or" instead of an "and". Nevertheless, this changes may be more readable than using negations.

Good catch @jorgepiloto .


Handle timezone-aware columns

I get this exception when I try to update any object in Flask-Admin where one column is timezone-aware:

TypeError: can't compare offset-naive and offset-aware datetimes

Full traceback here: https://gist.github.com/anonymous/31a48affeaa586fa8997

+1

That's good to have but I'm not sure what's best way to make it happen.
All form management is done by WTForms and, as fair as I know, WTForms works with naive Date/DateTime objects. It is possible to create custom DateTimeField that will produce timezone-aware dates and make Flask-Admin use it instead of vanilla DateTimeField.
But I'm not sure how to make it part of Flask-Admin - field should configurable enough to support different timezones for different admin views.

+1. Im using postgresql + sqlalchemy. If column in postgres defined as timestamp with timezone i get this error. But only on edit. On create it works fine. And no matter if i set in my model DateTime(timezone=False) or DateTime(timezone=True). It still doesnt work with the error in first message. But only when edit, so i guess edit just should act same as create. Basically right now it makes impossible to use flask-admin with timestamp with timezone fields.

Assuming the user is in the same timezone as the server seems like a bad idea. That's what it's doing by default when it creates the new record.
I think if we're going to solve this the right way, we need to add better timezone support to flask-admin.
This might require:

Adding an option that tells flask-admin to start storing datetimes in the database as UTC.
Configuring the datetime picker to set form values as UTC and display as the user's local timezone.
Changing the list view to display datetimes in the user's local timezone using moment.js.
If DateTime(timezone=True), wtforms will need to process the datetime field value as a timezone-aware UTC datetime.

I made some progress on adding timezone support here (it's still a work in progress): https://github.com/flask-admin/flask-admin/compare/master...pawl:issue_772?expand=1
Here's my approach so far:

If timezone=true on a datetime or time SQLA column, it will use a different WTForms field type.
The new WTForms field uses a new WTForms widget with an extra hidden input. That hidden input stores the UTC datetime that gets submitted for processing. The non-hidden text input that gets styled doesn't have a name attribute, so the non-UTC datetime value in that input don't get processed.

The idea is to allow displaying timezone-aware datetimes in the user's local time. But... now I'm stuck trying to find a way to do this that doesn't look hacky (currently it loads one value, then replaces it after the page finished loading).
I may end up taking a different approach and either display timezone aware datetimes as UTC or display datetimes with timezone offsets + processing that format. I'm also having 2nd thoughts on adding timezone support to time or date columns. Without both the date and time, there can be ambiguity when it comes to daylight savings time.

+1. I'm facing the same issue as what aldarund has mentioned above in November.

This has been a long time pain for us. It doesn't make sense to use naive datetime's anywhere outside of the scope of a single function.

This is my workaround so far. It prevents using of the timepicker as bootstrap-datepicker doesn't seem to understand anything about timezones. With some minor tweaks to it I think there could be a half-usable solution here. But at least this allows the form to function.
Note: this requires you have python-dateutil installed.
from wtforms.ext.dateutil.fields import DateTimeField

class PlainDateTimeField(DateTimeField):
    pass

form_overrides = {
    'created_at': PlainDateTimeField
}

form_args = {
    'created_at': dict(display_format='%Y-%m-%d %H:%M:%S%Z')
}

Looks like this:

Please refer to the gist on how I have fixed this issue. It takes care of the error in the edit form and saves the timestamp in UTC.
https://gist.github.com/hmanicka/82834f2e0df016947730dbaf19d25c28

Can we please revisit this? It is a major pain point and the solution in the last message does not work.


Make floor plan generation scriptable

We'll want to handle bad embarks:

[ ] No water
[ ] No wood
[ ] Aquifer
[ ] Glacier
[ ] Evil biome
[ ] Caverns too close to the surface


Notion integration

Hi,
While this is not really an issue, is it possible that we will see an integration with Notion?
Thank you for the awesome tool :D.

Great idea, I really don't use Notion, but I'll investigate if I can do it this weekend.


build: Exclude .github from apache-rat-plugin check

Which issue does this PR close?

Closes #31.
Rationale for this change

What changes are included in this PR?

How are these changes tested?

Merged. Thanks.


Recoverable applications lister (Ramen, not ShioRamen)

New requirement from Sumant on 29-April-2022:
Provide a list of applications to recover from given a pointer to an S3 bucket and what is needed to access it.
TODO:

[ ] make pull requests (to be referenced later) non-draft
[ ] Reviewed
[ ] Merged

Fixed in ShioRamen by PR #43


Feat: add demo playground (v3)

The version 3 looks fine in your personal website: https://orestbida.com/ Great job!

@carlosmonterocanabal thank you, you can also check out the live playground page to see how other layouts look like.


Setup build server

[x] wercker use ember-try
[ ] travis build

related: https://github.com/kategengler/ember-try/pull/41


Can anyone provide a brief tutorial on how to use the BinaryWriter and BinaryReader?

This, or point me to a resource to learn how to use Barbosik's BinaryWriter implementation.
I want to be able to use it to send more data in the packets but I have no idea where to begin!

For example, what is binary protocol 4 vs 5 vs ... 11?
I understand that writing to binary rather than sending plaintext or raw JSON is much more efficient, but how the underlying binary is translated is murky to me.
Also, how would the client know to translate the message back into JSON (or at least make use of it?)

Stop using socket.io EL OH EL.

This, or point me to a resource to learn how to use Barbosik's BinaryWriter implementation.

I understand that writing to binary rather than sending plaintext or raw JSON is much more efficient, but how the underlying binary is translated is murky to me.

To even know how to use it, you need to have a solid understanding of how the values get encoded in the first place. Here's a summary of the important.

What bytes are
The integer system (numbers without decimals)

Signed and unsigned integers
Integer sizes and their minimums & maximums

The float system (numbers with decimals)

Their precision degrades exponentially
Their more accurate but expensive to decode variant - decimals

The string system

Number represents a string
1 byte 1 character system - ASCII
UTF-8 and UTF-16

Once you learn that, you should know what JavaScript bitwise operators do.
Briefly, ~~x, x >> 0 and x >>> 0 floors a number, x & y is used for flags

I want to be able to use it to send more data in the packets but I have no idea where to begin!

Look at the source code.

For example, what is binary protocol 4 vs 5 vs ... 11?

As agar.io kept being updated, the binary protocol structure changed. None of them are compatible with each other so a public server might want to include support for multiple of them, which is what Barbosik did, and we are continuing on, albeit in a very slow pace.

Also, how would the client know to translate the message back into JSON (or at least make use of it?)

writer.writeStringZeroUtf8(JSON.stringify(object)) to encode,
JSON.parse(reader.readStringZeroUtf8(object)) to decode.
Yes, socket.io makes everything easier and more sophisticated, but being sophisticated always ends up slower. Don't use others' tools unless it's a bare bones driver, like a mysql client.

Don't flood my comments :b: UKA

:(

:)

Yeah socket.io is super easy but I feel if I want to scale it its pretty much impossible.
so by using ws you won't be sending JSON files, ever? I want to definitely transition to binary transmission.
Thanks for posting the links, appreciate them. I was already sort of familiar with bit-shifting, but the protocols just didn't make sense until you pointed them out as exclusive to agar.io.

Also one more thing, why do you scramble X and Y? I see the documentation says it prevents anti-raga?

 We just offset the coordinates by an arbitrary number for all clients to make clients harder to know where cells actually are.

No response in 5 days, closing for now.

I ended up writing my own Reader and Writer for arraybuffers if anyone cares.


HSEARCH-2070 Using TypeMetadata API instead of IndexedTypeDescriptor API

Pull request updated with a fix following @gunnarmorling 's advice.

thanks @gsmet and @fenrhil I'm mostly away today and tomorrow but will make sure these faceting fixes are included in the next 5.5.2

Hi @Sanne!
Nice, thanks. I pushed some additional fixes for @DavideD to HSEARCH-2069.

looks great, I'm merging it. Thanks a lot!


⚠ Bofrost.de has degraded performance

In 4dc15c5, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 753 ms

Resolved: Bofrost.de performance has improved in c2ca31f after 6 minutes.


By default, omit prepack and postpack scripts

Successful run: https://github.com/discourse/ember-route-template/actions/runs/6142626479/job/16664671253
Output: https://github.com/discourse/ember-route-template/blob/cbe87bf62a36637e43caba408749b7e83373298a/package.json

Thank you!

Published as v2.0.0


Schema to Go code generator

Please describe what you would like to see in this project
We need a go:generate code generator that takes an Arcaflow YAML structure loaded using this library (see schema package) and generates Go code (structs, etc) as well as the type definitions in Go code. The result should be something like this and this.
Please describe your use case
Go does not have enough information in its typing system to be able to construct an Arcaflow schema (no annotations). We need the code generator to turn the typing information into usable Go code without duplicated effort.

Validated the skeleton of a working hardcoded setup with the arca-team.
Moving on to the actual parsing.

Splitting the feature into two smaller PRs:

PR#1 for the typedef file
PR#2 for the shema file

PR#1 submitted for review.


NOT issue but question.

Is possible in the future the configuration of app will be made automatically ?

I got the error "TensorFlowLiteC framework not found" even though I copied this to the ios folder of the plugin in .pub-cache. How did you get through this OP?

@khue3012 did you try running dart pub get like #141 suggests?

@CaptainDario Yes. I ran that command so the plugin folder showed up. I copied the TensorflowLiteC.framework folder into the ios folder in the plugin folder. But it didn't work.

Do I need to manually link something by using Xcode?

@khue3012 I didn't need to link something with xcode. I ran
Dart pub get
Flutter pub get
Pod install

And than copied the tensorflow lite framework into the tflite_flutter directory (both the dart and flutter directory)
Than it worked.

Thank you for your quick response. I will try it.
Have a nice day man.

@khue3012 I had a lot of problems getting this pulgin working (on iOS, on android it was a painless process).
I think if we find a stable solution to this, it would benefit everybody.
For me it is now working but the MetalDelegate always throws an exepction #152 . So maybe my setup is also not correct.

Yes. I think this plugin is great but it lacks some instructions for people who don't have lots of experience with native iOS/Android.

Thanks for your suggestions @CaptainDario and @khue3012, I will work on updating the documentation, especially on the iOS side.


Add pagination to place list

Adds

Pagination bundle
Paginator to Places list

Modifies

Remplaces table design by card design on Places list

Rebased and merged manually, commits are in the master branch.


Console-based Log Analyzer for Amun Honeypot

what we have finished:

extendible framework
an example of log parser
documentation of usage

I have carefully reviewed your work. Well done and thx you guys so much!!!! But one thing I wish to comment out: you can further develop a more detailed documentation to help guide future developers. Again, really appreciate your work!!!!


Strange results when t_eval has repeat values

Thanks for the great package!
Summary:
I'm trying to switch over from using scipy's odeint to NumbaLSODA as it seems much better! Sometimes the t_eval array I supply has repeated times at the end and this produces some strange output. This was okay with odeint though, would it be possible to allow it for NumbaLSODA too? I think it is trying to integrate an infinitely small timestep rather than just evaluating the same timestep multiple times.
Details:
I'm trying to add this to LEGWORK so that we can speed up how we evolve the orbits of binary stars due to gravitational wave emission. However, if the evolution gets too close to the merger then LSODA produces a bunch of warnings about the timesteps being too small (since the derivatives get so large). To avoid this, I set any timesteps that are too close to the merger equal to the last allowable timestep - this leads to repeated values and the weird results that I see. (I don't just trim the array since I do this in a 2D array with a collection of binaries and so each individual binary might have a different length array which would wouldn't work in a 2D array I think.)
MWE:
Evolve binary evolution due to gravitational wave emission using Peters (1964)
from numba import cfunc
import numba as nb
import numpy as np
from NumbaLSODA import lsoda_sig, lsoda

@cfunc(lsoda_sig)
def de_dt(t, e, de, data):
    e_ = nb.carray(e, (1,))
    beta, c_0 = nb.carray(data, (2,))
    de[0] = -19 / 12 * beta / c_0**4 * (e_[0]**(-29.0 / 19.0) * (1 - e_[0]**2)**(3.0/2.0)) \
        / (1 + (121./304.) * e_[0]**2)**(1181./2299.)

ecc = 0.5
beta = 2.47e22
c_0 = 1.677e10
t_merge = 1.81e17

funcptr = de_dt.address
u0 = np.array([ecc])
data = np.array([beta, c_0])
t_eval = np.array([0.0, 0.1, 0.2, 0.5, 0.8, 0.9, 0.9, 0.9]) * t_merge

ecc_evol, _ = lsoda(funcptr, u0, t_eval=t_eval, data=data)
print(ecc_evol)

This outputs
[[ 5.00000000e-01]
 [ 4.83244343e-01]
 [ 4.64840518e-01]
 [ 3.95592532e-01]
 [ 2.82404708e-01]
 [ 2.15245963e-01]
 [-1.36311572e+57]
 [-1.36311572e+57]]

So you can see that as soon as the timesteps repeat, you get this hugely small number (and eccentricity should be between 0 and 1)

Oh and also hi @Nicholaswogan, I just started as a grad student at UW astronomy, maybe I'll see you in a meeting at some point 😄

@TomWagg Thanks for the issue. I think i fixed it: 7748fdf549cb11fa994527485d520d1f0d330eda . Your test case now produces
[[0.5       ]
 [0.48324434]
 [0.46484052]
 [0.39559253]
 [0.28240471]
 [0.21524596]
 [0.21524596]
 [0.21524596]]

This must be right? Once you confirm I will publish a new version on pypi.
Welcome to UW! Ya, I'm sure we will see each other. I hang out with Vikki Meadows group pretty often. LEGWORK looks awesome.

@Nicholaswogan awesome, this looks perfect, thank you for fixing so quickly!!
Thanks! Ah nice, she seems great! 😁 Thank you, glad you like it ☺


@Test in extensions throws fatal "is inaccessible due to 'private' protection level"

Description
When I try to implement a @Test inside a final class extension I'm getting this error:

Stack

  /var/folders/vz/vhht0c115gn1vwx3bjzv19xr0000gn/T/swift-generated-sources/@__swiftmacro_3TCK11TCKTestCaseC7example4TestfMp_.swift:25:21: error: '$s3TCK11TCKTestCaseC7example4TestfMp_
13funcexample__fMu_' is inaccessible due to 'private' protection level
                in: $s3TCK11TCKTestCaseC7example4TestfMp_13funcexample__fMu_,
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/carlos/Documents/Projetos/Swift/Reactor/Sources/TCK/TCKTestCase.swift:67:2: note: in expansion of macro 'Test' here
        func example() {}
        ^~~~~~~~~~~~~~~~~
/var/folders/vz/vhht0c115gn1vwx3bjzv19xr0000gn/T/swift-generated-sources/@__swiftmacro_3TCK11TCKTestCaseC7example4TestfMp_.swift:2:32: note: '$s3TCK11TCKTestCaseC7example4TestfMp_13
funcexample__fMu_' declared here
private static nonisolated var $s3TCK11TCKTestCaseC7example4TestfMp_13funcexample__fMu_: Any.Type {
                               ^
/Users/carlos/Documents/Projetos/Swift/Reactor/Sources/TCK/TCKTestCase.swift:67:2: note: in expansion of macro 'Test' here
        func example() {}
        ^~~~~~~~~~~~~~~~~
/var/folders/vz/vhht0c115gn1vwx3bjzv19xr0000gn/T/swift-generated-sources/@__swiftmacro_3TCK11TCKTestCaseC7example4TestfMp_.swift:29:31: error: '$s3TCK11TCKTestCaseC7example4TestfMp_
13funcexample__fMu1_' is inaccessible due to 'private' protection level
                testFunction: $s3TCK11TCKTestCaseC7example4TestfMp_13funcexample__fMu1_
                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/Users/carlos/Documents/Projetos/Swift/Reactor/Sources/TCK/TCKTestCase.swift:67:2: note: in expansion of macro 'Test' here
        func example() {}
        ^~~~~~~~~~~~~~~~~
/var/folders/vz/vhht0c115gn1vwx3bjzv19xr0000gn/T/swift-generated-sources/@__swiftmacro_3TCK11TCKTestCaseC7example4TestfMp_.swift:7:31: note: '$s3TCK11TCKTestCaseC7example4TestfMp_13
funcexample__fMu1_()' declared here
@Sendable private static func $s3TCK11TCKTestCaseC7example4TestfMp_13funcexample__fMu1_() async throws -> Void {
                              ^
/Users/carlos/Documents/Projetos/Swift/Reactor/Sources/TCK/TCKTestCase.swift:67:2: note: in expansion of macro 'Test' here
        func example() {}
        ^~~~~~~~~~~~~~~~~

Expected behavior
Test got declared and work correctly
Actual behavior
Swift compiler throws "<test_name> is inaccessible due to 'private' protection level"
Steps to reproduce

Create a final class:

public final class Suite {}

Add some extension to it:

public extension Suite {
  @Test
  func example() {}
}

swift-testing version/commit hash
branch: main/1afab67
Swift & OS version (output of swift --version ; uname -a)
swift-driver version: 1.87.1 Apple Swift version 5.9 (swiftlang-<IP_ADDRESS>.108 clang-15<IP_ADDRESS>)
Target: arm64-apple-macosx14.0
Darwin Carloss-MacBook-Pro.local 23.0.0 Darwin Kernel Version 23.0.0: Fri Sep 15 14:41:43 PDT 2023; root:xnu-10002.1.13~1/RELEASE_ARM64_T6000 arm64

I am unable to reproduce this issue on the October 3 main toolchain snapshot. The test in question compiles correctly, but there is a warning about the redundant use of public on the extension.
Note that the Swift 5.9 toolchain is not supported. See our Getting Started article outlining what you need to do to get up and running.

[!NOTE]
There is a bug in current main toolchains that prevents building and testing swift-testing. Use the most recent 5.10 toolchain instead for now.

If you are still seeing the issue when using the most recent 5.10 toolchain, please feel free to reopen this issue. We'll need to know if you're building in Xcode or at the command line, and it would be helpful if you could link us to a complete repository that demonstrates the issue (since I wasn't able to reproduce the issue with just the code provided above.)
Thanks!


TypeError: Object.defineProperties called on non-object

Using the latest version from npm i'm experiencing this issue when I connect to a local db and I try to create a database.
Here the code used:
db.exists(function(error, exists){
if(error)
 console.log(error);
else if (exists)
 console.log("ok");
else{
db.create();
db.save('_design/test',{
  all: {
   map: function(doc){
    if(doc.name) emit(doc.name, doc);
   }
}
});

Running this code once I receive this error:
/node_modules/cradle/lib/cradle/response.js:126
return Object.defineProperties(obj, descriptor);
^
TypeError: Object.defineProperties called on non-object
at Function.defineProperties (native)
at Object.extend (/node_modules/cradle/lib/cradle/response.js:126:19)
at Request._callback (/node_modules/cradle/lib/cradle.js:187:18)
at Request.callback (/node_modules/cradle/node_modules/request/main.js:119:22)
at Request. (native)
at Request.emit (events.js:70:17)
at Request. (/node_modules/cradle/node_modules/request/main.js:521:16)
at Request.emit (events.js:67:17)
at IncomingMessage. (/node_modules/cradle/node_modules/request/main.js:483:14)
at IncomingMessage.emit (events.js:88:20)

HI, I'm getting the same problem reported on this thread.

cradle: last from npm
node: v 4.2.3
couchdb: v1.6.1

Error stack:
/home/jcataluna/devel/testMediaVida/testnoapi/node_modules/cradle/lib/cradle/response.js:133
    return Object.defineProperties(obj, descriptor);
                  ^

TypeError: Object.defineProperties called on non-object
    at Function.defineProperties (native)
    at Object.__dirname.extend (/home/jcataluna/devel/testMediaVida/testnoapi/node_modules/cradle/lib/cradle/response.js:133:19)
    at Request._onResponse [as _callback] (/home/jcataluna/devel/testMediaVida/testnoapi/node_modules/cradle/lib/cradle.js:221:20)
    at Request.init.self.callback (/home/jcataluna/devel/testMediaVida/testnoapi/node_modules/cradle/node_modules/request/request.js:198:22)
    at emitTwo (events.js:87:13)
    at Request.emit (events.js:172:7)
    at Request.<anonymous> (/home/jcataluna/devel/testMediaVida/testnoapi/node_modules/cradle/node_modules/request/request.js:1035:10)
    at emitOne (events.js:82:20)
    at Request.emit (events.js:169:7)
    at IncomingMessage.<anonymous> (/home/jcataluna/devel/testMediaVida/testnoapi/node_modules/cradle/node_modules/request/request.js:962:12)
    at emitNone (events.js:72:20)
    at IncomingMessage.emit (events.js:166:7)
    at endReadableNT (_stream_readable.js:905:12)
    at doNTCallback2 (node.js:441:9)
    at process._tickDomainCallback (node.js:396:17)

I'm creating databases dynamically so I really need it working ;)
Regards.

Hi @jcataluna ,
Would you mind also test with the PR I sent
https://github.com/flatiron/cradle/pull/260

I had this problem. But for me it turns out that the error was coming from auth before db creation.


add mediator implementation by Networks Synergy

@mikelytaev -- can you please update the DCO on this?  DCO - Developer Certificate of Origin - https://github.com/apps/dco.
You can ignore the build failure -- we need to fix that.
Thanks


How to handle multiple canvas?

You will want to create multiple instances of p5 to do this. For an easy example take a look at http://www.joemckaystudio.com/multisketches/

Thankyou


hreview backcompat

php-mf2 does not do hreview backcompat see: http://microformats.org/wiki/h-review#Parser_Compatibility
Example
HTML

URL

Current Output
    "items": [
        {
            "type": [
                "h-review"
            ],
            "properties": []
        }
    ]

Expected output
    "items": [
        {
            "type": [
                "h-review"
            ],
            "properties": {
                "url": [
                    "http://example.com"
                ]
            }
        }
    ]

A more completed, contrived example:

 Review of Product

 5 stars on a scale of 0 to 5

review summary

Review description

 Reviewed by John Doe March 17, 2018

 Tagged: Reviews

 Permalink

Parsed:
{
    "items": [
        {
            "type": [
                "h-review"
            ],
            "properties": {
                "rating": [
                    "5"
                ],
                "worst": [
                    "0"
                ],
                "best": [
                    "5"
                ],
                "name": [
                    "review summary"
                ],
                "published": [
                    "2018-03-17"
                ],
                "content": [
                    {
                        "html": "
Review description
",
                        "value": "Review description"
                    }
                ],
                "author": [
                    {
                        "type": [
                            "h-card"
                        ],
                        "properties": {
                            "name": [
                                "John Doe"
                            ]
                        },
                        "value": "John Doe"
                    }
                ]
            }
        }
    ],
    "rels": {
        "tag": [
            "/tag/reviews"
        ],
        "self": [
            "/review-permalink"
        ],
        "bookmark": [
            "/review-permalink"
        ]
    },
    "rel-urls": {
        "/tag/reviews": {
            "text": "Reviews",
            "rels": [
                "tag"
            ]
        },
        "/review-permalink": {
            "text": "Permalink",
            "rels": [
                "self",
                "bookmark"
            ]
        }
    },
    "debug": {
        "package": "https://packagist.org/packages/mf2/mf2",
        "version": "v0.4.1",
        "note": [
            "This output was generated from the php-mf2 library available at https://github.com/indieweb/php-mf2",
            "Please file any issues with the parser at https://github.com/indieweb/php-mf2/issues"
        ]
    }
}

fn, photo, and url not correctly parsing as properties of p-item h-item
rel=tag and rel="self bookmark" not being upgraded to properties

I'll work on the fix.

Also, I believe the expected output of @kartikprabhu's original example is:
"items": [
    {
        "type": [
            "h-review"
        ],
        "properties": {
            "item": [
                {
                    "type": [
                        "h-item"
                    ],
                    "properties": {
                        "url": [
                            "http://example.com"
                        ]
                    },
                    "value": "http://example.com"
                }
            ]
        }
    }
]

#234 appears to have mostly fixed this, though I noticed the reviewer backcompat to p-author h-card is missing the name in the properties, so still need to address that:
    "items": [
        {
            "type": [
                "h-review"
            ],
            "properties": {
                "rating": [
                    "5"
                ],
                "worst": [
                    "0"
                ],
                "best": [
                    "5"
                ],
                "name": [
                    "review summary"
                ],
                "category": [
                    "reviews"
                ],
                "published": [
                    "2018-03-17"
                ],
                "content": [
                    {
                        "html": "
Review description<\/p>",
                        "value": "Review description"
                    }
                ],
                "author": [
                    {
                        "type": [
                            "h-card"
                        ],
                        "properties": [],
                        "value": "John Doe"
                    }
                ]
            }
        }
    ]

I have a PR in progress for this.


makeBaseApp.pl, eregen, etc. not available in default nix develop enviornment from template

Description
When following the tutorial, I ran into some issues with commands not being available
To reproduce
Follow the tutorial here: https://epics-extensions.github.io/EPNix/ioc/tutorials/streamdevice.html
Specifically:
$ nix flake new -t 'github:epics-extensions/epnix' my-top
[...]
$ cd my-top
$ nix develop
[...]
$ makeBaseApp.pl -t ioc example
makeBaseApp.pl: command not found

Hey there! Thanks for your interest in EPNix!
Just to be sure, did you ran the makeBaseApp.pl command inside the nix develop environment?
I just tried these exact instructions, and it worked for me.
Could you check that:

epics-base is contained in your path
whether the makeBaseApp.pl command is inside the /nix/store/....-epics-base-.../bin directory
whether running /nix/store/....-epics-base-.../bin/makeBaseApp.pl -h directly works

Another potential lead: are you on a x86_64-linux machine?

Yes, I ran makeBaseApp.pl in the nix develop shell

Epics-base is not contained in path either
makeBaseApp can be run from /nix/store directly
This works.

I'm on x86-64-linux-NixOS via WSL2, so there may be some difficulties there. I'll see if I can recreate the same issue in a "real" NixOS vm.

hmmm, that's weird. EPnix was previously used on WSL2, and it did work. I don't have a WSL2 system on hand right now so I can't test it immediately.
You have epics-base in the nix store, so it did build it to make the development environment.
Could you show the result of nix print-dev-env from your top?


Fix Typos in Developer Website's Open Source Section on Front Page

Fix Typos in Developer Website's Open Source Section on Front Page
🚀 Goal
We want to ensure our website's different pages don't have any mistakes.
🤔 Background
This issue focuses specifically on fixing typos across our general website pages. Your contributions here make a difference!
🤝 How You Can Contribute
We've broken it down into specific typos for you to address. Every contribution counts!
🔑 Tasks and Acceptance Criteria
❗You must only assign yourself one task at a time to give everyone a chance to participate.❗
You may assign yourself your next task after your current task is reviewed & accepted.

/open-source Page

[ ] Remove extra space before comma in "Apache License 2.0 ,"

Make sure all tests are passing.

🌟 Resources

Developer Website

Getting Started

Comment ".take" on this issue to get assigned
Fork the repository and create a new branch for this task
Follow the tasks outlined above
Submit a pull request with your changes
Respond to any feedback during the review process

Questions?
If you have any questions or need clarification, please comment on this issue or join our Discord community.
Happy coding! 🎉

Please assign me this task as a part of hacktoberfest.
.take

.take


add block_reader interface

Apache Beam is interested in using fastavro instead of apache/avro, but needs an interface like I've added here, for seeking to and reading individual record-blocks in a file.
I've published this as fastavro-blocks:<IP_ADDRESS> to test.pypi.org (cf. blocks-test-pypi branch, which has a commit on top of this PR changing the name and version), and used that in a Beam branch here.
Presumably it can't go into Beam until it's in real pypi, so that will have to wait until the next release here, though those seem to happen quite frequently 🎉😀
In this change I introduced a block-iteration interface, and had the standard record-iterator wrap it. I also did a few random tweaks to make _read{.pyx,_py.py} diff more cleanly against one another.
I'm new to Cython and haven't figured out when _read_py.py would be used instead of _read.pyx; a previous version of this PR failed Travis' pypy builds due to a bug I introduced in _read_py.py, so I guess it's covered?

Thanks for the PR. I'll try to take a look soon and give a full review. One note is that we will want a test or two using the new functionality, but I can help with that if needed.

Thanks, I'm adding some tests and fixing bugs found along the way; I'll bump this when that's done

OK I think this is in a better state; I added a test, and cleaned up the Block.{offset,size} semantics so that they refer to actual file- or memory-positions and distances, whether the data is compressed or not.

Thinking I'll expand the tests to verify reading the records from a temp-file (currently they just read from an in-memory buffer)

Thanks for all the feedback! I think I addressed everything, lmk what you think!

Thanks! This looks great. Merging this and will get a version up on pypi shortly.

Nice job, but how can I directly get the length of each block?

@swordspoet IIRC, Block.bytes_ is the uncompressed bytes, and Block.size is the compressed size, does that get you what you need?
(seems like they could use some docstrings…)

@ryan-williams thank you, what I want is how many records in each block, and I want to improve speed by directly getting the numbers of record in each block. I have tried your method and got the size of block.

@ryan-williams thank you, I found thishttps://github.com/fastavro/fastavro/blob/0.21.4/fastavro/_read_py.py#L567

@swordspoet @ryan-williams I created an issue to better document those attributes (as well as others) when I find some time. Hopefully this will make it easier for others in the future.


MudDataGrid: Add new edit dialog options

Description

The DataGrid edit dialog has hard coded Title ("Edit") and Button names ("Save", "Cancel"), offers only one line for textfield editing and every column requires an entry when editing. This new feature enables customized Title and Button names implemented via a new class "DataGridEditDialogOptions", which holds the properties "TitleValue", "SaveValue" and "CancelValue". The options "EditDialogTextFieldLines" and "EntryRequired" were added to "column.cs" to enable customized editing textfield size and to determine wheter or not a specific column requires an entry.
So far:

Changes applied:

How Has This Been Tested?

One new UnitTest has been added, and an existing one has been modified in file "DataGridFormOptionsTest.razor". Added new test "DataGridDialogEditOptionsTest()" to check whether applied changes work and modified existing test "DataGridDialogEditTest()" to check whether existing default values still work.
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Checklist:

[x] The PR is submitted to the correct branch (dev).
[x] My code follows the code style of this project.
[x] I've added relevant tests.

I would need the enhancements of this pull request. Can somebody please merge it? Can I help in some way?
Franz


Azure Machine learning Experiment cancel requested for 40hrs +

I wanted to cancel a launched experiment but it was stuck in cancel request for 40hrs + as can be seen below...
The process inside the experiment was still running, logs were being updated.
The code inside was a pretty long running while (Model training) loop, however it was not doing anything special like swallowing Interrupt or Terminate signals so one would expect it to be able to be terminated in a usual way.
Is there any reason why this would happen?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 889f9925-878f-e2a0-262a-240e1611bce2
Version Independent ID: 9349cb37-8107-3d5a-98d1-9a45419c64de
Content: Track, monitor, and analyze runs - Azure Machine Learning
Content Source: articles/machine-learning/how-to-track-monitor-analyze-runs.md
Service: machine-learning
Sub-service: core
GitHub Login: @swinner95
Microsoft Alias: shwinne

Hello @JordanSimba Thanks for the feedback.
I have assigned the issue to the content author for further review and update the doc as appropriate.

Hi @JordanSimba
This issue repository is primarily intended for documentation-based issues. To get the best help for technical issues, you should create a support ticket in the Azure portal:

Navigate to the Azure portal.
Select the support question mark (?).
Select Help + support
Select New support request
Include a link to this Issue URL in your support request.

For a more detailed guide, see Create an Azure support request

Closing since it is not a documentation related issue. @JordanSimba please feel free to open another issue if you run into issues with the documentation. #please-close


Add helm chart for Apache Druid

Description

This patch added a helm chart for Apache Druid.

This PR has:

[x] been self-reviewed.

[x] using the concurrency checklist (Remove this item if the PR doesn't have any relation to concurrency.)

[x] added documentation for new or modified features or behaviors.
[x] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
[x] added or updated version, license, or notice information in licenses.yaml
[x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
[x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
[x] added integration tests.
[x] been tested in a test Druid cluster.

cc @maver1ck @AWaterColorPen

Do we need a tutorial like the tutorial of docker ( docs/tutorials/docker.md ) ?

Hi, @FrankChen021 . Thanks for your comment. I also considered whether to publish the doc, for example, put the helm/README.md file in the docs/tutorials directory. However, although this Helm Chart can deploy on the K8S cluster directly, there are still a few things that need to be completed before the document is published on the official website. For example: upgrade the Druid image version, support HDFS cluster deployment, release the Release version of Druid's Helm Chart by configuring Github Action, and etc. But these things are difficult to complete in a short period of time, so we should finish them in these follow-up PRs.

Hey @asdf2014, have you sorted out the potential license issue before you merge this PR? I don't see it anywhere. As Julian suggested in https://lists.apache.org/thread.html/ra93200245a27fb77ff0a1cfc17d21b7efbbb263ae422d6d2cc5ade11%40<dev.druid.apache.org>, we may need to go through a process similar to the IP clearance for podling projects.

Hi, @jihoonson. I didn't notice it, let me take a look. Thank you very much for reminding.

Hey @asdf2014, have you sorted out the potential license issue before you merge this PR? I don't see it anywhere. As Julian suggested in https://lists.apache.org/thread.html/ra93200245a27fb77ff0a1cfc17d21b7efbbb263ae422d6d2cc5ade11%40<dev.druid.apache.org>, we may need to go through a process similar to the IP clearance for podling projects.

I think we should consider reverting this until we get this resolved, after reading through https://incubator.apache.org/ip-clearance/, and looking at https://incubator.apache.org/ip-clearance/tika-tika-helm.html which appears to be a similar helm chart donation for another project, where I see linking to a voting thread, etc.

unrelated nitpick, i think putting this stuff in distribution/helm/, which is also where the Docker stuff lives, makes more sense to me than top level helm/

btw, in general I think it makes sense to maintain it here, so +1 on the idea (even though I know very little about helm)

Hi, @clintropolis . Thanks for your comments and support. As far as I know, the license of helm/charts is
Apache License 2.0. Maintaining Apache Druid’s own Helm Chart in the Apache Druid repo should not have copyright issues. I heard about the process just mentioned by @jihoonson for the first time, so I’ll look at it. In addition, Apache Superset has actually done this, and we can get more details through the link https://github.com/apache/superset/tree/master/helm/superset.

@asdf2014 my understanding is that we still need to highlight were the code came from, just like we do for other Apache License 2.0 code we include or borrow.
Also, has this chart been tested? Does it still work?

I haven't closely checked the IP clearance or other ASF processes to adopt codes from outside the project after graduation. But my 2 cents is we probably need a PMC vote for adopting.

Hi, @xvrl. Thanks for your comment. I agree with you, the follow-up related documents will be improved. I also mentioned before that this Druid Helm Chart is completely usable, and it has been tested several times last week.

Hi, @jihoonson. Thank you for your comments. It was previously maintained by @maver1ck @AWaterColorPen and I. Currently, https://github.com/helm/charts/tree/master/incubator/druid cannot be maintained. So, we want maintain it here, just like other Apache projects did, such as Apache Superset. Indeed, I agree with your proposal. Let's raise up a PMC vote. If it really fails to pass the vote, we can revert this PR ASAP. Could you please help initiate the vote?

@asdf2014 I'm not saying we should not accept this. I'm saying this doesn't seem the right process. Have you read the ASF IP clearance doc? The IP clearance is not just about the license. All codes in the ASF projects are owned by the ASF and the IP clearance process is required to make this part clear. The IP clearance doc explicitly says

This is for projects and PMCs that have already been created and are receiving a code donation into an existing codebase. Any code that was developed outside of the ASF SVN repository and our public mailing lists must be processed like this, even if the external developer is already an ASF committer.

I think we should revert this PR and go through the IP clearance process because this was developed outside of ASF. Please go back to the dev thread for this issue and add more details there (not here) for people like me who don't know much about the helm chart such as

What is the current status of the project?

Does it reflect the most recent release of Druid?
How is it being tested?

Why is it best to host the helm chart in the druid repo? What alternatives have you considered?
After migration, what is your plan for automatic testing?

I don't think this is an exhaustive list. Please add details as much as possible.

@jihoonson I see. And I agree with your suggestion. Let us follow the process of IP clearance to eliminate potential problems. I will do my best to provide answers to these questions. I have discussed it with other maintainers, and there are many improvements to it that cannot be contributed to the open source community. So, let's unify the maintenance of Druid's Helm Chart as soon as possible. Thanks a lot for your patience.


[Typo] Blog Link

I can't access the website correctly when I click the blog link in the README.
So I checked the official website and found the link appears to be case-sensitive.
It should be https://www.outeai.com/blog/outetts-0.1-350m
rather than https://www.outeai.com/blog/OuteTTS-0.1-350M.

Fixed it. Now you can use both links. Thanks for letting me know!


Add debug sentry endpoint

Context
Our goal is to have confidence production errors are being tracking for this app. Currently the prod environment isn't appearing at all. No errors, no transactions, just some session info for the dev and test environment.
However, when we make the AP UI throw an error we can see errors come through!
We follow this sentry guide for node to force an error from the frontend to test the connection: https://docs.sentry.io/platforms/node/guides/expressc
This will confirm our only slight theory is that receiving an error forces the environment to be recognised.
I really wanted to add a simple request test for this but fell into a rabbit hole. Is this the type of thing that's possibel without using jest and mocks? I wanted to write the following and would be eager to know if something like it is possible:
describe "sentry"
  it "returns a 500 error"
    expect{ get "/debug-sentry" }.to raise_error(Error('Test Sentry error thrown.'))

Screenshots of UI changes

Release checklist
As part of our continuous deployment strategy we must ensure that this work is
ready to be released at any point. Before merging to main we must first
confirm:
Pre merge checklist

[ ] Have all changes to any dependencies been deployed to both preprod and
production in a backwards compatible way? (This includes our API,
infrastructure, third-party integrations and any secrets or environment variables)
[ ] Has a required data migration been included in this change or been done in
advance?

Post merge checklist
Once we've merged we now need to release this through to production before
considering it done.
Find the build-and-deploy job in CircleCI
and work through the following steps:

[ ] Has the product manager asked to provide approval for this feature?

[ ] Has the product manager approved?

[ ] Manually approve release to preprod
[ ] Verify change released to preprod
[ ] Manually approve release to prod
[ ] Verify change released to production

Should a release fail at any step, you as the author should now lead the work to
fix it as soon as possible.

So you could definitely test this in the integration tests - visit this endpoint and this is what you see on screen. And if you moved this into a controller you could unit test that an exception is actually being thrown (though that may be overkill). Happy to chat in the morning about this


Add filters + search to admin hackathons page

So it seems like full-text search won't work perfectly with names like TestHacks vs Test Hacks. I still think it'd be better cause we're able to use indexes, but maybe there's a way to add automatically separate names by capital letters and include those in the tsvector.


cmd/govim: properly handle LSP client ShowMessage

In https://github.com/myitcv/govim/pull/444 we simply log the fact that the server asked us (the client) to show a message. If the message is important we should show it, preferably as a popup.

Hi @myitcv, I would like to work on this. It would be very helpful if you could push me in the right direction. How do you decide if a message is important or not?

Thanks for the interest @dragfire

It would be very helpful if you could push me in the right direction

Very happy to. First thing to say: I think it's probably worth waiting for https://github.com/myitcv/govim/pull/467 to be merged before tackling this. Because it's the upgrade of gopls that causes these calls to be made.

How do you decide if a message is important or not?

But you can, by all means, have a poke around. Taking a look at the client method:
https://github.com/myitcv/govim/blob/956c1ebcc75bed45653d44f26109d9bca1a772ad/cmd/govim/gopls_client.go#L21-L23
the second parameter has a type defined as follows:
https://github.com/golang/tools/blob/master/internal/lsp/protocol/tsprotocol.go#L1731-L1742
The Type field is in effect an enumeration:
https://github.com/golang/tools/blob/master/internal/lsp/protocol/tsprotocol.go#L3579-L3597
So in the first instance I think we want to show a notification popup when we receive a message of Error type.
What do I mean by a notification popup? I've proposed a potential approach in https://github.com/myitcv/govim/issues/318#issuecomment-513582991. That results in something like the following:

See :help popup_create-arguments for more popup options.
Please feel free to ask any questions you like. Having new contributors helps to ensure that our docs (or lack thereof right now) for (new) contributors are up-to-date, sufficient etc.

@myitcv Thank you so much. I will play around with this a bit. Looking forward to contributing to this repo.

We just merged https://github.com/myitcv/govim/pull/487, so please feel free to have a play around against 6961210ff30a5862594ff04dde9f3a5f62120ae4

@myitcv I just implemented a popup_create channel call for Error MeesageType. But I am not sure how to test this method. Please have a look at: https://github.com/dragfire/govim/blob/master/cmd/govim/gopls_client.go#L21-L34
How do I test that this particular method is working? And if I want to see the popup being displayed on my local machine, how to do that?

@dragfire

But I am not sure how to test this method

Great question.
We could wait for https://go-review.googlesource.com/c/tools/+/194018 to land in gopls, then create a test that triggers this error.
Whilst we wait for point 1, we could in the meantime simulate a message from gopls. The way to do this would be to create a test-only Vim function, similar to GOVIM_internal_DumpPopups:
https://github.com/govim/govim/blob/74faeda6b8893a4bf040becc4218426da282c7c5/cmd/govim/test_config.go#L36
This function would need to asynchronously trigger a call to:
https://github.com/govim/govim/blob/74faeda6b8893a4bf040becc4218426da282c7c5/cmd/govim/gopls_client.go#L21
We can then call that Vim function from a test, for example see:
https://github.com/govim/govim/blob/74faeda6b8893a4bf040becc4218426da282c7c5/cmd/govim/testdata/hover.txt#L13
Please feel free to start a PR. It's often easier to add comments to code.


az iot edge set-modules command doesn't give the output and keeps on loading. It doesn't get terminated.

az iot edge set-modules --hub-name test-qa-Hub --device-id  testagh  --content D:\onboard\module.json
This command sets a module in the IOT edge device, but it doesn't exit. The command keeps on showing 'Running' status for hours, evenafter completed the module setting in the iot edge device.
It was working fine in local machine when i eecuted via poweshell.
This problem happens in jenkins. Tried commands in with both CLI plugin and windws powershell build step. Both shows same behavior.

Configuration is done as above.

Same error occurred for me. I find a workaround as I used batch windows command and run the Azure cli command in it. I used service principal to login to the batch command. Environment variable can be used as %variable_name% in the command.
@ECHO OFF ECHO Running batch command for deployment  az login --service-principal --username 1278afc7-3494-42be-b444-9d7c87aa957e --password **************** --tenant ********************* && az storage account create --name %storageAccountName% --resource-group %resourceGroupName% --sku Standard_LRS --location %location% && az functionapp create --name %fnAppName% --consumption-plan-location %location% --storage-account %storageAccountName% --resource-group %resourceGroupName% --functions-version 3 --assign-identity Pause
Here I have made my build as parameterized and using the variable from pipeline variable in the command. If you need my help, let me know.


Multi-positional eligibility

From reviewing the code, I can't tell if the optimizer handles multi-position eligibility

I do not believe so. I looked at this in the past and ended up writing my
own from scratch.
On Mon, Mar 13, 2017, 6:31 PM Eric Truett<EMAIL_ADDRESS>wrote:

From reviewing the code, I can't tell if the optimizer handles
multi-position eligibility
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/DimaKudosh/pydfs-lineup-optimizer/issues/6, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AFpAH359ZN7PEZ7w5JKf0g-4qpWN5Y9pks5rldHDgaJpZM4Mb8kf
.

What constraints did you add? No more than 1 of a player ID?

If you add a player multiple times to the pool for different positions, you could create a constraint like this.
for pid in set([p.id for p in players]):
prob += sum([x[player] for player in players if player.id == pid]) <= 1

I'll look into it on weekends.

Kudos to the optimizer that was created, it really is a great solution, efficien, is very customizable, and fits my exact needs. I just wanted to second that I am still encountering this issue where a player with multiple positions will be added to the lineup as a member of both those positions.
A work around I found was adding the player twice to the pool, assigning them a unique id, then creating a constraint in the optimizer listed above (this was inserted into the optimizer function):
for pid in set([p.id for p in players]):
                prob += lpSum([x[player] for player in players if player.id == pid]) <= 1

What version of library you use? It should be fixed in latest version.

I'm running version 1.1.2 of the library and python 2.7

Can you provide info about site and sport and players list sample?


Add link to Usability page on CC Search page

At the bottom of the CC Search page we reference our forthcoming Usability page.
Once https://github.com/creativecommons/creativecommons.github.io-source/issues/267 is complete, we need to update the CC Search page to update the text at the bottom of the page and add the link in.
Note: this ticket is blocked until https://github.com/creativecommons/creativecommons.github.io-source/issues/267 is done.

Now, this can be unblocked :)

done, thanks!

Can I take this up?

Sure @ritesh-pandey

If I understand correctly, updated text will be
Visit our Usability page for more details on how to participate.

Yes @ritesh-pandey


webhooks: Do not log RateLimited exception to webhook logger.

In authenticated_rest_api_view (but not webhook_view) the rate
limiting happens inside the try / catch block that potentially
reports errors to the webhook logger (and thus Sentry, if so
configured).  Rate limits are RateLimited exceptions, which do not
descend from JsonableError (the RateLimitMiddleware deals with
transforming them), and as such are, by default, logged; .
Skip logging RateLimited exceptions to the webhook logger.

Do we want to instead change these to be a subclass of JsonableError?

Totally plausible; I don't know the history of why it isn't.  @mateuszmandera, any insight there?  Looks like it used to inherit from PermissionDenied, but doesn't any more, which may leave open the path to making it a JsonableError.

I'm not sure why originally it wasn't a JsonableError (presumably due to needing to inherit from PermissionDenied as mentioned), but making that change seems like a promising idea based on my quick looking into it. I'll work on a PR for making it happen.

Sounds good; @mateuszmandera maybe just push that version to this PR thread to keep the discussion in one place?

Upon second inspection, this can't be done easily, this is how  RateLimitedMiddleware processes the exception:
    def process_exception(self, request: HttpRequest,
                          exception: Exception) -> Optional[HttpResponse]:
        if isinstance(exception, RateLimited):
            # secs_to_freedom is passed to RateLimited when raising
            secs_to_freedom = float(str(exception))
            resp = json_error(
                _("API usage exceeded rate limit"),
                data={'retry-after': secs_to_freedom},
                status=429,
            )
            resp['Retry-After'] = secs_to_freedom
            return resp
        return None

What cannot be expressed via JsonableError:

 data={'retry-after': secs_to_freedom} because data from a JsonableError is constructed from the data_fields class attribute:

    @property
    def data(self) -> Dict[str, Any]:
        return dict(((f, getattr(self, f)) for f in self.data_fields),
                    code=self.code.name)

So retry-after can't be accomplished through this - we would have to change to retry_after or make the method above more fiddly to deal with this special case.
2. resp['Retry-After'] = secs_to_freedom - JsonableError would have to be extended to allow setting headers.
We can extend JsonableError to deal with these problems, but at this point, it's probably better to just keep the exception processing in RateLimitMiddleware?

Aha, OK -- thanks for digging!  Can you push a commit to the end of this branch that adds a comment to the RateLimited class (and/or the RateLimitedMiddleware) that distills your above discoveries, for future reference?

I'd probably prefer to just try to figure out solving those problems; as a subclass of JsonableError, we can write whatever custom code we want for it in its data method (including continuing to have that process_exception handler).
What I'm thinking is basically this:
class RateLimited(JsonableError):
    def data(...):
        result = super(...)
        result['retry-after'] = ...
        return result

One reason to me for pushing on this is that we try really hard to avoid untested code in the core serving code path, which the current implementation has.

Unless I'm misunderdasting, this actually was tested already, in test_external.py, but I pushed a commit doing the switch to JsonableError, because it does seem to make things cleaner.

Thanks looks like a great cleanup!  Pushed without my first commit -- feel free to merge once it passes tests.


Rebuild this swiper with reanimated library and Typescript

if someone want to contribute for building a clean one, contact me and lets code...

Whenever you have PRs ready open them and tag me and I’ll take care of it


Expose z/OS Connect Designer through Wazi DevSpaces Workspace

Description
The new release of z/OS Connect now supports running the Designer UI through Wazi Devspaces. Our current automation stands up z/OS Connect Designer and Server on the boot node, which I do not expect our clients to use in this way. We should ensure that all usage of z/OS Connect happens through the OCP interfaces.
Potential Solutions

Reach out to z/OS Connect team and understand proper usage and documentation with Wazi DevSpaces
Deploy z/OS Connect Designer more "natively" through a Deployment resource and any applicable Routes.

Additional Context
Snippet for a Devfile that can bring up a z/OS Connect Designer container
schemaVersion: 2.2.0
metadata:
  name: wazi-devspaces
  namespace: <dynamically-generated?>
attributes:
  controller.devfile.io/devworkspace-config:
    name: devworkspace-config
    namespace: openshift-operators
  controller.devfile.io/storage-type: per-user
components:
  - attributes:
      controller.devfile.io/merge-contribution: true
  - container:
      endpoints:
        - exposure: public
          name: zosconnect-ui
          path: /zosConnect/designer/
          protocol: http
          targetPort: 9080
      env:
        - name: ZCON_DESIGNER_PROJECT
          value: /projects/zopeneditor-sample/apiProjectWorkspace/apiProject/start
      image: icr.io/zosconnect/ibm-zcon-designer:3.0.70
      memoryLimit: 4927M
      mountSources: true
      sourceMapping: /projects
    name: zosconnect

Documentation for using z/OS Connect with Wazi DevSpaces can be found here.

Additionally, automation for installing the CBSA app on a z/OS environment can be found here:
https://github.ibm.com/mathieu-dalbin/CICS-Bank-Sample-Application/tree/AWS-Demo/automation
This could be something that is driven through an automated tekton pipeline, or directly from Wazi DevSpaces in the built-in IDE.

@ivandov PR is raised with the changes. Please review.

@ivandov In the PR raised, even though a devfile (with cbsa project) from GitHub is referenced for use in the Wazi Devspaces, but a different devfile (without cbsa project) could be seen created by default in the Wazi Devspaces (see below screenshot).

So I am suspecting the devfile can't be changed from OpenShift, as it is a property of Wazi Devspaces.
The possible solution what I could think of is;

Put the devfile in GitHub and retrieve its raw url
After the Wazi Devspaces is created using the automated Ansible playbooks, Create Workspace in Wazi Devspaces by manually importing using raw url (see below screenshot)

Note : The devfile with cbsa project should be kept in a specific location in GitHub. Users should be informed to import it using raw url in their Wazi Devspaces.

Please share your thoughts on this approach.
In the screenshots shared above, you could see two Workspaces.

wazi-devspaces ---> Created using Ansible playbook tasks, referring the devfile in GitHub. But It doesn't have cbsa project, even though the devfile in GitHub has it.
wazi-devspaces-new ---> Created manually from Wazi Devspaces UI by importing using raw url of devfile in GitHub. It has the cbsa project.

Able to expose cbsa application UI from a Workspace in Wazi Devspaces.
PR is raised and merged with the changes. Closing the issue.


please help me understand!

This is not a issue, it's a question.
Why is it possible to have  apps like nautilus, or firefox and many of the other WSL2 integrated apps, but i can't simply lunch gnome-session?!?!?
It doesn't make any sense? Why?!?
I mean you had to go trough some really complex processes to make that happening right?
So please, answer me. WHY?! Why did you guys spent time and effort to make sure only parts of linux can be launched. I mean it's obvious you did something purposely to stop people from using a full gnome desktop in windows, but why?!?1
Are you guys afraid that people will lunch gnome desktop and just used the incredible  slick and fast working desktop instead of windows? Cose of course they will.  do you guys know what's the number 1 thing people,  who are just normal users,  are trying to do when they learn about WSL? Yeah, that's right! they try to lunch Gnome. And they cannot! Why?!? I mean you guys made linux work in windows. That is great! Amazing! So if you went and did it, why than broke it ?! Just so you have people wasting their time ??!
This is unbelieve!

Have you seen these two issues? Does wslg support i3wm, sway, xfce4 (in the wslg repo) and We have gui apps, but when will we be able to run gnome-session??


ENH: Semantic Type for NCBI protein-taxonomy data

What's new

Closes #66
Semantic type for a directory containing tree files

nodes.dmp
names.dmp
prot.accession2taxid.gz

Why is this useful? This Semantic Type can be used as optional input to the build-custom-diamond-db action, thereby allowing the user to obtain a database with taxonomy features.

Run it locally
Assuming you have a working environment, run the following.

First, clone the repo and checkout the PR branch:

git clone git@github.com:bokulich-lab/q2-types-genomics.git
gh pr checkout 67

Now let's get you the data to try out the Semantic Type.

cd wherever_you_want_to_download_the_data_to

Be aware that the next command will download ~15 Gb of data

mkdir ncbi_tax_data
wget ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdmp.zip -P ncbi_tax_data
wget ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/accession2taxid/prot.accession2taxid.gz -P ncbi_tax_data

Now make an artifact with the new semantic type

qiime tools import --input-path ncbi_tax_data --output-path ncbi_tax_data.qza --type ReferenceDB\[TaxonomyNCBI\]

Alternatively, you can run the tests for this Semantic Type (nothing will be downloaded).

cd <local_path_to_q2-types-genomics>
pytest

@misialq

That all LGTM 🏅 I think, though, that this change will require updating the classify-kaiju code in q2-moshpit since you move the format used by that action - could you please double check and maybe open a small PR there to fix that, if needed? Thanks!

I believe there is no update required. The object that is used in classify-kaiju is KaijuDBDirectoryFormat which contains the file formats (NCBITaxonomyNodesFormat and NCBITaxonomyNamesFormat) that were moved with this PR. Now NCBITaxonomyNodesFormat and NCBITaxonomyNamesFormat are imported into the module where KaijuDBDirectoryFormat is defined, instead of being defined in that same module. Only KaijuDBDirectoryFormat is imported by q2-moshpit, not NCBITaxonomyNodesFormat and NCBITaxonomyNamesFormat, so I don't see anything that breaks.


Logstash restart breaks logs flow

Hi,
I have configured logstash and logspout to send logs from machine A to machine B. Everything works fine, but restarting logstash breaks the logs flow. All logs are still visible using http output and logspout stdout says logstash: write udp: connection refused. That's expected: logstash is booting up. Restarting the logspout container makes it send the logs again.
How can I make logspout reconnect without restart?

We are having the same issue. Our solution was to modify the plugin to just die at connection error (and then have a custom logspout container that waits for the connection to be available in a bootstrap)

We're having the same problem. Any good workaround out there? Watchdog solution, restart logspout if it happens?

@sege this was our change: https://github.com/mdx-dev/logspout-logstash/blob/master/logstash.go#L58-L59

I'm seeing the write udp connection error as well except for me, it seems to be happening randomly.  Is there a good way to troubleshoot why it happens or pinpoint what exactly is causing the failures?

I have changed the behavior to exit with code 1 on write failures, if this helps. See 6b2fe61.

I should have some time to test this today.

So it looks like the restart on non zero exit status is working.  However it is also restarting when it hits an oversized packet.  Here's what the log looks like:
2016/06/23 17:11:30 logstash:write udp <IP_ADDRESS>:36305-><IP_ADDRESS>:5001: write: message too long
Is this expected?  I think everything is okay, the logspout containers just restart quite a bit.

It is expected, as every error is treated equally for now. If you can come up with a strategy on how to handle too large messages a PR would be nice, but I think the subject is a bit tricky with UDP. Any thoughts?

I'll close this for now as no solution has been suggested.


Ambient sounds cannot be heard after launching the MR portal

Bugs
Playground: https://www.babylonjs-playground.com/#PCY1J#54
Repro steps:

Make sure you have some working headphones connected to the PC. Also make sure the MR portal is not currently running.
Go to the playground scene above, then wait until you hear the music playing.
Click the VR headset icon. This will launch the Windows MR Portal.

Result: The music ceases to be audible. It no longer plays in your original headphones, and even if your headset has built-in headphones, the music isn't audible there either. Moreover, hitting escape to exit VR mode doesn't bring the music back. The browser thinks the music is playing, though (it shows the speaker icon in the browser tab).
Expected: The music should be audible in either the original headphones or the headset, depending on your MR portal settings.

Unfortunately this is more an Edge but than a Bjs bug :(

Turns out, I can repro this in Chrome:

Go to the playground I linked in Chrome.
Open the MR Portal through the start menu in Windows.

This will silence the audio as well. Interestingly, I don't see the same issue with other audio sources, for example, Spotify and YouTube in the browser.

Interesting. I repro (Actually I'm losing the sound AND my keyboard when I do that :))
They seem to mess with the WebAudio API somehow
I bet Youtube and Spotify should rely on Audio tag and not webaudio

Even this demo does not work: https://webaudiodemos.appspot.com/midi-synth/index.html

Ok, I figured it had something to do with WebAudio. It's clear that it's not a Babylon-specific issue. Do you have any contacts on the team that maintains the MR Portal? It would be good to raise this issue with them.

Pinging @davrous  who may know better than me

Ping... @davrous


Rename uv pip command

Thanks for your work on uv and ruff! I would like to suggest changing the uv pip subcommand name. uv and pip are different tools. uv is not necessarily going to implement the same interface as pip. It feels a bit crummy to write a replacement to another tool and then take its name. I would suggest renaming the subcommand to pkg or something similar that's more descriptive and won't lead to unnecessary confusion. Leaving pip as an undocumented alias makes sense, but I don't think it should be the default name for the package management subcommand.

Closing as duplicate of https://github.com/astral-sh/uv/issues/1657

Note the intent is only for this to be a compatible interface and we do intend to have entirely separate interfaces for all of the same functionality but with a new workflow. There are many tools with pip in the name, e.g. pipx, pip-tools, etc. Nonetheless, we agree though changing the name might make sense and are considering a few options.


No trumpet inhands

reported by @"Grandma#0105"

Happy to find this if someone wants to assign it.
If there's any other missing art let me know and I can do it in a big oner


FreeBSD expects nullptr when trying to load symbols from "self" via dlsym

dyncall will accept a nullptr for dlLoadSym.  Need to figure out how to best handle the special case.

I was able to fix this issue by adding a special case in vm::terp::register_foreign_function for FreeBSD.  The other platforms require a valid library pointer, even for self.  FreeBSD wants nullptr for self and a valid pointer for everything else.


Add public access orga setting

resolves #4204

After activating the setting once and then reloading the setting looks deactivated again.

If the global setting is disabled please hide the icons "public" -> please hide the indicator that a meeting is public
Please hide the login as a guest user button in the login mask

Also move the setting "Meetings can be public" to the Superadmin settings below: "Enable chat globally"
connected with this please hide the setting for non-superadmins


Setup simple model from scratch

includes .nc

There's now a Docker test that checks this from a model saved by HYDROLIB-core. It fails with:

** WARNING: MDU setting "Wrihis_temperature = 1" asks to write temperature to the output his file, but no temperature is involved due to MDU setting "Temperature = 0". So we set "Wrihis_temperature = 0"  and do not write temperature to his file.
Dimr [2021-12-09 10:30:23.75548] #0 >> kernel: MDU setting "Wrihis_temperature = 1" asks to write temperature to the output his file, but no temperature is involved due to MDU setting "Temperature = 0". So we set "Wrihis_temperature = 0"  and do not write temperature to his file.
** WARNING: MDU setting "Wrihis_salinity = 1" asks to write salinity to the output his file, but no salinity is involved due to MDU setting "Salinity = 0". So we set "Wrihis_salinity = 0" and do not write salinity to his file.
Dimr [2021-12-09 10:30:23.76163] #0 >> kernel: MDU setting "Wrihis_salinity = 1" asks to write salinity to the output his file, but no salinity is involved due to MDU setting "Salinity = 0". So we set "Wrihis_salinity = 0" and do not write salinity to his file.
** WARNING: TimeSplitInterval invalid, disabling time partitioning of output. Got: 0X
Dimr [2021-12-09 10:30:23.77235] #0 >> kernel: TimeSplitInterval invalid, disabling time partitioning of output. Got: 0X
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0
[unset]: write_line error; fd=-1 buf=:cmd=abort exitcode=1
:
system msg for write_line failure : Bad file descriptor
** WARNING: readMDUFile: [geometry] changevelocityatstructures=0 was in file, but not used. Check possible typo.
Dimr [2021-12-09 10:30:23.77570] #0 >> kernel: readMDUFile: [geometry] changevelocityatstructures=0 was in file, but not used. Check possible typo.
** WARNING: readMDUFile: [geometry] changestructuredimensions=1 was in file, but not used. Check possible typo.
Dimr [2021-12-09 10:30:23.77592] #0 >> kernel: readMDUFile: [geometry] changestructuredimensions=1 was in file, but not used. Check possible typo.
** WARNING: readMDUFile: [output] wrishp_enc=0 was in file, but not used. Check possible typo.
Dimr [2021-12-09 10:30:23.77629] #0 >> kernel: readMDUFile: [output] wrishp_enc=0 was in file, but not used. Check possible typo.
** WARNING: readMDUFile: [output] ncnounlimited=0 was in file, but not used. Check possible typo.
Dimr [2021-12-09 10:30:23.77645] #0 >> kernel: readMDUFile: [output] ncnounlimited=0 was in file, but not used. Check possible typo.
** WARNING: readMDUFile: [output] ncnoforcedflush=0 was in file, but not used. Check possible typo.
Dimr [2021-12-09 10:30:23.77657] #0 >> kernel: readMDUFile: [output] ncnoforcedflush=0 was in file, but not used. Check possible typo.
** WARNING: readMDUFile: [output] wrimap_flow_analysis=0 was in file, but not used. Check possible typo.
Dimr [2021-12-09 10:30:23.77676] #0 >> kernel: readMDUFile: [output] wrimap_flow_analysis=0 was in file, but not used. Check possible typo.
** WARNING: readMDUFile: [output] waterlevelclasses=0.0 was in file, but not used. Check possible typo.
Dimr [2021-12-09 10:30:23.77689] #0 >> kernel: readMDUFile: [output] waterlevelclasses=0.0 was in file, but not used. Check possible typo.
** WARNING: readMDUFile: [output] waterdepthclasses=0.0 was in file, but not used. Check possible typo.
Dimr [2021-12-09 10:30:23.77701] #0 >> kernel: readMDUFile: [output] waterdepthclasses=0.0 was in file, but not used. Check possible typo.
** WARNING: could not open '/home/runner/work/HYDROLIB-core/HYDROLIB-core/tests/data/output/docker/Boundary_net.nc'
Dimr [2021-12-09 10:30:23.77970] #0 >> kernel: could not open '/home/runner/work/HYDROLIB-core/HYDROLIB-core/tests/data/output/docker/Boundary_net.nc'
** ERROR  : Observation file 'ObservationPoints.ini' not found!


Add Standalone Production config

Following the excellent work by Anthony I split the nginx into a proper config production which doesnt use the vite development server.
I still have a few open questions/remarks I want to discuss:

Env File, imo we should put the .env into version control, just an example of it. My prefered solution would be to create a file called .env-example and .gitignore the .env file.
The development version as provided right now is not really working, as the vite can't access the wss for HMR. I would ask the question if we should really use nginx for the dev setup. Happy for your input here.
Healthchecks: @bendhouseart Your changes to the healthchecks in the compose file, didn't work for me. What was the point behind your changes?
Startup scripts (dev.sh and prod.sh) I see the benefit of those files (git submodules & key generation) but also would try to look for a streamlining of the storage of environment variables (script vs. .env) and the duplication of code between the dev and prod file. + I don't completly understand why the npm dependencies are installed in that script and not in the Dockerfiles of the corresponding containers. Option might be to merge dev and prod into a start.sh with more flags.
Nginx config file, I think this a good starting point, not sure tho how far we can go with this as setups are quiet different (e.g. we have dont have our cert key encrypted and as a result don't have a ssl_password_file. Thus the nginx fails. Maybe remove the config from the code repo and put it into the documentation as an example instead?
The api still runs in dev mode more or less. Didn't completly get what the start.sh is used for here (maybe nowhere at all?). Might have time to tackle that in the next few days.

Happy for your input.
Best,
Tim

Env File, imo we should put the .env into version control, just an example of it. My prefered solution would be to create a file called .env-example and .gitignore the .env file.

Yeah, that makes sense, but I prefer example.env ;), but I think it's worth adding an if [ ! -f .env ]; then cp example.env .env fi somewhere to handle things (adding this to my branch right now).

The development version as provided right now is not really working, as the vite can't access the wss for HMR. I would ask the question if we should really use nginx for the dev setup. Happy for your input here.

I'd like to commit to nginx always being run b/c that makes the development and production environments as similar as possible (barring how brainlife.io deploys things). That is to say I think it's simpler to have 2 nginx configuration files and 1 env file then to have multiple docker compose files, but that solution while being simpler to me may not be the best path forward.

Healthchecks: @bendhouseart Your changes to the healthchecks in the compose file, didn't work for me. What was the point behind your changes?

It was required as the health checks were failing when reaching out to localhost after we got nginx up and behaving. Since we were all in on nginx it seemed like a good idea to just go with it in the compose too.

Startup scripts (dev.sh and prod.sh) I see the benefit of those files (git submodules & key generation) but also would try to look for a streamlining of the storage of environment variables (script vs. .env) and the duplication of code between the dev and prod file. + I don't completly understand why the npm dependencies are installed in that script and not in the Dockerfiles of the corresponding containers. Option might be to merge dev and prod into a start.sh with more flags.

Yeah, I had some similar remarks about the run/build process, but I think it's helpful to keep in mind that this is not how ezbids is deployed on brainlife.io. Technically speaking, everything we've been doing has been "non-production" compared to how this project was originally scoped/intended to work.
I agree and will be working today to streamline the launch/build process to better leverage:

environment variables via .env and otherwise
making dev.sh a bit more explicit by renaming it launch.sh
adding more options to dev.sh/launch.sh  for production, rebuilding/installing npm after mounting etc.

Nginx config file, I think this a good starting point, not sure tho how far we can go with this as setups are quiet different (e.g. we have dont have our cert key encrypted and as a result don't have a ssl_password_file. Thus the nginx fails. Maybe remove the config from the code repo and put it into the documentation as an example instead?

Yeah when I requested my ssl certs I encrypted the keys, I'm not sure of a clever way around this within the nginx config. Before we think too much on this, could you try running the http version (point to nginx/development_nginx.conf) instead? My thinking is to use the http nginx enabled version for local development and the https nginx version for production going forward.

The api still runs in dev mode more or less. Didn't completly get what the start.sh is used for here (maybe nowhere at all?). Might have time to tackle that in the next few days.

No idea either, but I think consolidation is the word of the week for my branch.

Hey everyone, I've been working with @bendhouseart to improve the setup and deployment on production especially for a setup where client and handler run on different machines. I the context of that we:

Renamed dev.sh to launch.sh and moved all the configuration done via command line arguments to an .env file.
Created a docker-compose with nginx for production to allow easy setup with a default configuration.

We tested both setup on our local infrastructure. Please let me know if anything else is missing ot get this merged. (Not sure why the dev.sh file is marked as a conflict, this PR should remove the dev.sh file)
Best

Yeah, I went ahead an closed the original PR as they've converged now with this. Currently able to launch this in either "production" or "dev" mode and confident enough to say lgtm.