best practices for structuring nested experiment runs

Hi folks,
I was wondering if there's a way to structure experiments for each each individual choice of dataset and algorithm.
For instance, you could have something in your code base like this:
foreach dataset
    foreach algorithm
          run() and record experiments

Eventually since these runs are based on individual combo of dataset and algo you would want to have an experiment for each of them.
How would you go about doing that?
One very bad way I came up with is the following:
experiments = []
foreach dataset
    foreach algorithm
          experiments.append(Experiment('dataset + algorithm combo name'))

foreach experiment in experiments
     experiment.main
     def main():
         main code calling models and running train

Please let me know what's a better way of doing that. Thank you!

Well I think your basic idea is right. However, given that you want to run the same experiment just with different configuration (model, data) you can reuse the same experiment object and just update the config. It would look something like this
ex = Experiment("generic_experiment")

@ex.main
def run(dataset, model):
    ...

for dataset in datasets:
    for model in models:
        ex.run(config_updates={"dataset": dataset, "model": model},
               options={'--name': f"{dataset}_{model}")

For a more complete example you might look at Klaus' code https://github.com/Qwlouse/Binding/blob/master/run_evaluation.py

@JarnoRFB thanks for the reply. In the end  that's what I ended up doing but it was a bit more involved since I was using different files src.py & main.py. The src.py contains Ingredient and main.py contains the actual Experiment with the ingredients. One issue that I faced was that I was loading the config from yaml files inside the main.py uisng ex.add_config but then in the main.py I couldn't get those configs to be injected during run time, even though I had the ingredient.capture decorator in some of the methods. This bit through me off. Maybe I was doing it wrong?
The other thing that I would like to ask is about populating some files with data such as best validation loss or accuracy, etc. And maybe how to potentially add more data i.e. images/plots.
Let me give a bit of context.
In omniboard I see something like this:

How can I populate the Resources and Results columns? I've already figured out from the docs how to make the live plots with metrics like accuracy/loss but I'm not quite sure aobut:

How to populate the above columns
What the best way of keeping track of best model during training (I suppose you should record accuracy/loss of best model and checkpoint files), if so which fields should we populate? Could we just use the _run.info? Maybe that won't show up in the omniboard?
Are the tags meant to be used with version control? or other also other uses cases which you can point me to?

Thank you for your patience and apologies if I'm asking too many questions?

In the end that's what I ended up doing but it was a bit more involved since I was using different files src.py & main.py. The src.py contains Ingredient and main.py contains the actual Experiment with the ingredients. One issue that I faced was that I was loading the config from yaml files inside the main.py uisng ex.add_config but then in the main.py I couldn't get those configs to be injected during run time, even though I had the ingredient.capture decorator in some of the methods. This bit through me off. Maybe I was doing it wrong?

Sorry, but I cannot quite follow. A minimal code example would greatly help here.

The other thing that I would like to ask is how would anyone go about saving validation folds in each experiment. Should that be in different columns using _run.info? Or should each of them be a separate experiment, but then it would unnecessary populate many entries in the db. Is there a proper way to populate those folds each as a separate entry a.k.a row but under it's own experiment.

What exactly do you want to save from the validation fold. If it is just a metric, e.g. accuracy, why not save this into a metric of the run? You can call
_run.log_scalar("validation_fold_acc", acc)

for each validation fold.
It would also be nice if you could ask such general question under the python-sacred on stackoverflow so the answers remain more visible to the general public.

Sorry, but I cannot quite follow. A minimal code example would greatly help here.

I apologize for the confusion let me provide a MWE as you requested in order to make things clear
src.py
import sacred
ingred = sacred.Ingredient('default-params')
ingred.add_config('some/yaml/file')   <--- added config
ingred.add_config('another/yaml/file') <--- added config

class MyModel(object):
   @ingred.capture
    def __init__(self):
        do some stuff...

main.py
import sacred
from src import MyModel, ingred
ex = sacred.Experiment('test-exper', ingredients=[ingred])

@ex.main   <--- this also works as capture decorator
def main(param1, param2):  <--- if I pass my params they are not recognized, only if I access them through _run
    MyModel()

What exactly do you want to save from the validation fold. If it is just a metric, e.g. accuracy, why not save this into a metric of the run?

Yup that's doable but when you examine through omniboard I think it shows the validation loss of the last  training epoch and not the best valid.
In other words since _run.log_scalar("validation_fold_acc", acc) always puts a counter I am not sure whether the values are appended for each count or if they are overwritten.

It would also be nice if you could ask such general question under the python-sacred on stackoverflow so the answers remain more visible to the general public.

Thanks for the pointer, wasn't aware of it, from now on I'll post related stuff there.

Yup that's doable but when you examine through omniboard I think it shows the validation loss of the last training epoch and not the best valid.
In other words since _run.log_scalar("validation_fold_acc", acc) always puts a counter I am not sure whether the values are appended for each count or if they are overwritten.

I believe that if you do not set the step explicitly, it will append to the metrics array. If you want to see the current best validation metric, you could set it as a result. While the experiment is running with
_run.result = best_validation_acc

and for the final result by returning the result value from the main function. See also https://sacred.readthedocs.io/en/latest/collected_information.html#live-information. This way it would be displayed in omniboard in the result column.
On the ingredient issue I unfortunately cannot comment without looking a bit deeper into it. I have not really used ingredients myself. But do I see it right that you want to access parameters from the ingredient config in the experiment main function?

On the ingredient issue I unfortunately cannot comment without looking a bit deeper into it. I have not really used ingredients myself. But do I see it right that you want to access parameters from the ingredient config in the experiment main function?

Exactly, without having to again use ex.add_config or either _run, just by accessing the params in the captured method def main(params). It seems that the params are not injected when using ingredients? Although I might be wrong!

Hi @kirk86,
ingredients create their own namespace in the configuration, as if the values where part of a dictionary with the name of the ingredient. If you slightly modify your example to use a python-compatible name for the ingredient you can access it from there:
import sacred
ingred = sacred.Ingredient('default_params')
ingred.add_config('some/yaml/file')   # <--- added config
ingred.add_config('another/yaml/file') # <--- added config

class MyModel(object):
   @ingred.capture
    def __init__(self):
        pass # do some stuff...

import sacred
from src import MyModel, ingred
ex = sacred.Experiment('test_exper', ingredients=[ingred])

@ex.main   # <--- this also works as capture decorator
def main(default_params):
    param1 = default_params['param1']
    param2 = default_params['param2']
    MyModel()

@Qwlouse
thanks a lot. That's great!
I'll close the issue for now to keep things clean.

Well I think your basic idea is right. However, given that you want to run the same experiment just with different configuration (model, data) you can reuse the same experiment object and just update the config. It would look something like this
ex = Experiment("generic_experiment")

@ex.main
def run(dataset, model):
    ...

for dataset in datasets:
    for model in models:
        ex.run(config_updates={"dataset": dataset, "model": model},
               options={'--name': f"{dataset}_{model}")

For a more complete example you might look at Klaus' code https://github.com/Qwlouse/Binding/blob/master/run_evaluation.py

@JarnoRFB, I came up with a similar solution but I’m having problems passing the dataset as an argument for the config_updates. Since I’m using the MongoObserver, the whole dataset is being saved to the MongoDB.
Is there any way to pass data to the ex.run() method without touching the config? As a workaround, I thought to use a global variable to hold the dataset reference but I wonder there is a more elegant solution.

@pedropalb Sorry not quite sure what you mean. I guess in the example I meant dataset to represent a reference to dataset, e.g. a string identifying the dataset or a path to the data. Otherwise all datasets need to loaded in memory upfront. As you pointed out, putting an instantiated dataset in the config is not great, but I think this should be the case irrespective of the observer used.


Set selected values in initialization

Using the Email Contacts example here: http://brianreavis.github.io/selectize.js/#tabs
var REGEX_EMAIL = '([a-z0-9!#$%&\'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&\'*+/=?^_`{|}~-]+)*@' +
                  '(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?)';

$('#select-to').selectize({
    persist: false,
    maxItems: null,
    valueField: 'email',
    labelField: 'name',
    searchField: ['name', 'email'],
    options: [
        {email<EMAIL_ADDRESS>name: 'Brian Reavis'},
        {email<EMAIL_ADDRESS>name: 'Nikola Tesla'},
        {email<EMAIL_ADDRESS>    ],
    render: {
        item: function(item, escape) {
            return '
' +
                (item.name ? '' + escape(item.name) + '' : '') +
                (item.email ? '' + escape(item.email) + '' : '') +
            '
';
        },
        option: function(item, escape) {
            var label = item.name || item.email;
            var caption = item.name ? item.email : null;
            return '
' +
                '' + escape(label) + '' +
                (caption ? '' + escape(caption) + '' : '') +
            '
';
        }
    },
    create: function(input) {
        if ((new RegExp('^' + REGEX_EMAIL + '$', 'i')).test(input)) {
            return {email: input};
        }
        var match = input.match(new RegExp('^([^<]*)\<' + REGEX_EMAIL + '\>$', 'i'));
        if (match) {
            return {
                email : match[2],
                name  : $.trim(match[1])
            };
        }
        alert('Invalid email address.');
        return false;
    }
});

Is there anyway way to pass in whats going to be the selected value? I can set it afterwards but that would trigger the onChange event which is not what I want.

Easy...
$('#p_cliente_id').selectize({
items: [5],   //------------>>> place between the id keys that you need to initialize the select.
create: true,
maxOptions: 10000
});

5ece5f5d86ab617f finally documented that, 15 days ago.

Any positive news. Stuck with the same problem. I have a multi select html . The options are shown first . Then I use ajax to change the options in html multi select  but the selectize does not update?
Any suggestions on how to update selectize?

@sujeshkumaruv What you're probably looking for (which isn't really on topic for this issue) is the refreshOptions() API method.  $('#id').data('selectize').refreshOptions()

I can not make the items[] to work. I am trying like this http://laravel.io/bin/mGD7n
The id 11 is in the list that will be loaded, what am I doing wrong?
I am trying to populate the selectize input with some values when editing a record...
Thank you!

hello. anything on this yet?  need to pre-select multiple items when editing a previous entry.  Seems important.

Thanks @felonious !! That was the easiest solution I found! Using setValue and other stuff didn't work for me!


Transaction retries and jitter

These should be easy and safe.

We discussed offline. The default should be one retry with a random amount of backoff, say evenly distributed in [100..500) ms. Motivation is to make it easy to recover from unique constraints.
We should have a separate and independent strategy to recover from connectivity problems.

This is done


Truncated output of CsvDataWriter

When I was creating small app for converting large datasets from one CSV format to another, in sizes about 10 millions lines on the input and 500K rows on the output, I noticed that the CSV output created by CsvDataWriter was always, deterministically(!), truncated by about 100-1000 characters. Simply set each file was missing from half a line to about few lines.
I tried using proper usings and await syntax with the CsvDataWriter, tried to find any "Flush" API I might have missed, but the output was always the same, always trimmed on the same position.
Please check the CsvDataWriter implementation, it looks like it's not counting the required buffer properly and truncating the output sooner with the result of data corruption.

This is almost certainly due to not flushing/closing one of the streams properly. Show me some code the reproduces the issue.

Method I used for the serialization:
async Task WriteCsvToFile(string outPath, object[] o)
{
await using DbDataReader recordREader = o.AsDataReader();
var csvWriter = CsvDataWriter.Create(outPath);
await csvWriter.WriteAsync(recordReader);
}
The record structure I serialized had only two columns, one stringId, second string with object serialized as json, so the CsvDataWriter had to do lots of escapings.

This code doesn't compile. object[].AsDataReader() wouldn't write any records either, because the type  (T) would be object, which has no properties. So, I'm guessing your real code is more complicated than this example.

It compiles, targeting .net 6.0 the project uses Sylvan.Data.Csv 1.1.16 and Sylvan.Data 0.2.3.
I'm using the method for serializing array of anonymous types.

It compiles

"recordREader" vs "recordReader"
I don't believe you.

Ok, fixed the typo :)

Try changing:
var csvWriter = CsvDataWriter.Create(outPath);
to
using var csvWriter = CsvDataWriter.Create(outPath);

So update, you're right, that with that parameter object[] it serializes empty lines. Originally I had the serialization logic outside that method.
Also after changing it to using var csvWriter = CsvDataWriter.Create(outPath); it works 🎉
I'm testing in on my second laptop with real data that I cannot share here.
So it looks like it works when using the API properly, the only question is why there is no warning in VS that I should be using await using with that type? Usually VS is quite vocal when I miss using for a disposable type somewhere.

why there is no warning in VS

That's a good question, and I thought that the code analysis packages would provide this now, but apparently not.
If you want to have a helper method like this, it needs a generic type parameter (T) to allow the AsDataReader to discover the properties:
async Task WriteAsync<T>(string filename, IEnumerable<T> data) where T : class
{
    using var r = data.AsDataReader();
    await using var w = CsvDataWriter.Create(filename);
    await w.WriteAsync(r);
}

Glad to hear your problem is solved. If you find these libraries useful, please give the repository a star.


TypeError: r is not async iterable

"fetch-event-stream": "^0.1.3",

Getting this error in chrome and edge windows latest stable - (v123.0.2420.65)
On chrome canary 125.0.6384.0 works fine no errors.
	const response = await fetch(`${PUBLIC_BASE_PATH}/api/ai`, {
		method: 'POST',
                headers:{
                	'fb_api_key':PUBLIC_AI_KEY
                },
                body: JSON.stringify({
                    message:msg,
                    userId:$page.data.session.user.email,
                    aiType:PUBLIC_AI_TYPE,
                    conversationId:chatID || createConversationID(),
                    promptId:'default'
                })
            });

			if (response.ok) {
				console.log('response',response);
				let stream = events(response);
				for await (let event of stream) {
					console.log('<<', event.data);
					try {
						const jsonData = JSON.parse(event.data);
					} catch (error) {
						sRes.addStreamRes(PUBLIC_AI_CONFIG, chatID, `${event.data} `);
					}

				}
			}

Here is the response

Ah, shoot. Sorry about this. Didn't realize that @@asyncIterator for ReadableStreams was experimental & not-well supported in browsers.
I'll change that, either via #1 or through the linked commit (incoming). Do you mind testing it out locally for me, to verify it works for you?

Fixed published in 0.1.4

@lukeed this is great thanks @lukeed tested with .5 works well.


lacks a pull request template file

Pull Request Template
Type of Change

[ ] Bug Fix
[ ] Enhancement
[ ] Documentation Change

Description of Changes

Checklist

[ ] Part of GSSOC
[ ] Tested for any breaking changes
[ ] Other relevant checks completed

Related Issue

Screenshots

Contributor Agreement

[ ] I agree to follow this project's Code of Conduct
[ ] I'm a GSSOC'24 contributor
[ ] I want to work on this issue

i can fix it if assigned to me under gssoc !!!

assigning to you

issue fixed!!
@varshith257

@prajwaldp223 Make PR to this repo. Not to your forked repo and merge there

okay!! i will delete that

now?


Fix scene example

Since #9907 the generation starts at 1 instead of 0 so Entity::to_bits now returns<PHONE_NUMBER> (ie. u32::MAX + 1) as the lowest number instead of 0.
Without this change scene loading fails with this error message:
ERROR bevy_asset::server: Failed to load asset 'scenes/load_scene_example.scn.ron' with asset loader 'bevy_scene::scene_loader::SceneLoader': Could not parse RON: 8:6: Invalid generation bits

If that fixes an example I'mm all for it :D
That said ; isn't the root cause using to_bits()[^1] ? or something else ? in my opinion this complicates the ron format, which I'd like to make sense of easily, seeing these kind of "magic" numbers don't facilitate understanding.
[^1]: docs of to_bits states it shouldn't be used in serialization ? https://github.com/bevyengine/bevy/blob/main/crates/bevy_ecs/src/entity/mod.rs#L258-L259

Yep. This entire way of serializing entities is terribly flawed, but the example exists as-is so it's better if it actually works.
Certified jank :')


RXN files: structure is missing when "Paste from clipboard" or "Open from file" if reaction consists of two or more reaction arrows and structures.

Steps to Reproduce

Open Ketcher v2.5.2
Add a reaction with multiple (more than one reaction arrow) as shown in the image
Save file in MDL Rxnfile V2000 and MDL Rxnfile V3000
Paste files from clipboard or open saved files from saved directory.

Expected behavior
Structure is correctly opens from saved files. Keep only first reaction arrow and keep all structures (all intermediate structures should be products and the arrow is replaced by a plus)
Actual behavior
In MDL Rxnfile V2000 the arrow disappears, and when opening MDL Rxnfile V3000 the arrow and structure disappear
Attachments

Indigo/Bingo version
Indigo Toolkit
Version <IP_ADDRESS>-g86f1ef175-x86_64-linux-gnu-11.2.1
Additional context
Indigo issue: https://github.com/epam/Indigo/issues/870

While writing test in this task https://github.com/epam/ketcher/issues/3181 found out like it's reproducing again
At least for V2000 RXN files


support linting multiple trees

Add broccoli-merge-trees
Add an includePaths configuration option to specify paths to be linted in addition to the app tree.A
Add documenation for includePaths with special attention to supporting addon authors.

 Sorry I didn't get to this sooner. @billybonks what do you think!

This looks good to me 👍 , thanks for the pr @jamesarosen 💃


[PE-D][Tester C] Adding of multiple timeslots causes error

What: In the given data upon fresh start, Alex Yeoh has only 1 timing. Upon attempting to add another timing, the app shows error which deviates from deviates from what is stated in the user guide; The ability to have multiple timings.
How:

Send command edit 1 !t
Modify values to edit 1 timeslot: Saturday 3pm-5pm, Monday 1pm-2pm which should add one more timeslot
Error message about timeslot format displayed.

Error displayed

Labels: type.FunctionalityBug severity.Medium
original: ys112/ped#14

"edit" can be used to delete/modify existing timeslots but not adding new timeslots

@amanzainal I was able to add/delete/modify timeslots from my end.. Can u help me check if its working at your end when you are free, thanks!

This is caused by changes I made in editCommandParser to how grades and timeslots are parsed:
```
if (!Grade.isValidGrade(trimmedGradeString)) {
throw new ParseException(Timeslots.MESSAGE_CONSTRAINTS);
}
if (!Timeslots.isValidTimeslot(trimmedTimeslotString)) {
throw new ParseException(Timeslots.MESSAGE_CONSTRAINTS);
}

Previously without this line, if you were to edit using an invalid timeslot, it wouldn't show any message, but like it wouldn't accept it either. I think we just need to move the check for isValidGrade and isValidTimeslot in this loop instead

      String[] timeslotArray = trimmedTimeslotString.split(",");
        for (String timeslot : timeslotArray) {
            timeslotSet.add(new Timeslots(timeslot.trim()));
        }

@joelgoh1 Ohhh, I see! Most likely I had resolved this issue while I was editing other error messages for the edit yesterday. Thanks a lot!


Drop PoolMetadataFetchAttempts in removePoolMetadata

When removing pool metadata, there's no point in keeping
the last fetch attempts. In fact, this might delay re-syncing
further.

bors r+


Embed script secure origins error

runner:26 Uncaught (in promise) DOMException: Only secure origins are allowed (see: https://goo.gl/Y0ZkNV).
    (anonymous function) @ runner:26
        Promise (async)
    (anonymous function) @ runner:26
       runner-3.39.19.min.js:1

@remy this was related to the problem I emailed you about .. the ability to rerun the conversion to iframes.
We are currently re-injecting the script tag.  But it recently stopped working on the second injections.
We're not sure if this error is related, but thought to post it anyway.

The behavior can be seen at http://canjs.github.io/canjs/doc/guides/chat.html. If you navigate anywhere using the sidebar and back to Chat Guide, the embed request is made but not executed.

But this isn't a fatal error though, right?
I could add a guard to check if the page is on https to prevent it dropping into the code I guess.

It's likely this wasn't the cause of our issue - just something new that I noticed. I found a workaround mentioned here. Feel free to close the issue, though it would be nice to eliminate the console error.


Fix issue React-Easy

Description
Created a simple card component in React that displays a random image
Additional Context

Merged

This is an automated message from Fork, Commit, Merge [BOT].

Thank you for your contribution! Your pull request has been merged. The files have been reset for the next contributor.
What's next?
If you're looking for more ways to contribute, I invite you to check out my other projects. Just click here to find more. These projects contain real issues that you can help resolve. You can also check out the Influences section in the README to find more projects similar to this one.
Also please leave a star to this project if you feel it helped you, i would really appreciate it.
I look forward to seeing your contributions!


sql/spanner: Add start of spanner driver, implement basics of inspection

This adds the start of a Google Cloud Spanner driver implementation.

cc @rotemtam 🎉

Some basic cleanup done on my end, will take a look at your feedback tonight @a8m!


Run llvm-lines on the entire test suite

It would be nice to be able to use cargo llvm-lines to debug slow test compilation times too, but as far as I can tell it doesn't support counting the lines for cargo test.
This request is similar to #16 but for the entire test suite rather than an individual test, which might be easier (or harder) to implement.

The --test option also only appears to work on integration tests found in tests/, but it would also be nice to be able to run this on unit tests that might be found in src/ as well.


Warnings

I have this warning:
config.json: warning: philipslights: ignoring unknown key
    "platforms": [{
            "platform": "Hue",
            "name": "Hue",
            "host": "<IP_ADDRESS>",
            "users": {
                    "001788..........": "U7nB6rt............"
            },
            "heartrate": 5,
            "timeout": 5,
            "lights": true,
            "philipslights": true,
            "wallSwitch": true,
            "sensors": true

philipslights has been deprecated since Nov 2017, see https://github.com/ebaauw/homebridge-hue/releases/tag/v0.5.43.  See https://github.com/ebaauw/homebridge-hue/wiki/Configuration for the supported keys.


Maint/159 clean up

Issue #, if available:
#159
Description of changes:
This PR

refactors apmtracesource into otel-trace-source and otel-trace-raw-processor
fix other places that depends on apmtracesource
renames upper-case to string_converter for StringProcessor and exposes the boolean to switch between upper and lower case

Note: tested on sample app example
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Could you resolve the conflicts?

Create a Readme in the plugins::common subproject and cover the below plugins.

Bounded blocking
String converter
File sink
File source
Std source
std sink

Create a Readme in the plugins::common subproject and cover the below plugins.

Discussed offline and will do in a separate PR.

Also updated overview.md. Will check running the sample configuration

@kowshikn Made some changes to overview.md and verified all the sample configurations work as expected.


IK should have both iterative and symbolic approaches

https://homes.cs.washington.edu/~todorov/courses/cseP590/06_JacobianMethods.pdf

http://www.cs.cmu.edu/~15464-s13/lectures/lecture6/jlander_gamedev_sept98.pdf
http://www.d.umn.edu/~snorr/ee5351s14/computeIK-JT.pdf
http://www.cs.cmu.edu/~15464-s13/lectures/lecture6/jlander_gamedev_nov98.pdf
https://en.wikipedia.org/wiki/Coordinate_descent
https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.optimize.fmin_tnc.html


Fix indentation on ACME setup examples

The selector field needs to be on the same level as the solver type (either dns01 or http01).
What this PR does / why we need it:
Fixes invalid example config in the docs.
Release note:

Fix indentation on ACME setup examples

🤦♂ ergh, no wonder people have been having issues!
Thank you for taking the time to make this PR 😄
/lgtm
/approve
/ok-to-test
/cherrypick release-0.8


Update rdf4j-model, rdf4j-rio-api, ... to 5.0.1

About this PR
📦 Updates

org.eclipse.rdf4j:rdf4j-model
org.eclipse.rdf4j:rdf4j-rio-api
org.eclipse.rdf4j:rdf4j-rio-nquads
org.eclipse.rdf4j:rdf4j-rio-turtle

from 4.3.12 to 5.0.1 ⚠
Usage
✅ Please merge!
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

⚙ Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.eclipse.rdf4j" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "30 days" },
  dependency = { groupId = "org.eclipse.rdf4j" }
}]

labels: library-update, early-semver-major, semver-spec-major, commit-count:1

Superseded by #130


Add support for exo secrets vault

Some TODOs remain but this is a working implementation of authing via exo and making secrets available in the environment.

Thanks! I'll look more closely soon. A really brief look though raises one key question: Does all of this happen in the exo cli? Are there any APIs to expose from the exo kernel or workspace such that the GUI can interact with all this stuff too?

@BenElgar please ping me when the UI for this is stable and I'll follow up with a quick styling pass for e.g. the buttons.

@jwmza Unless Ben said otherwise, I think the UI is stable for this PR.


Slug update

Update slug to avoid usage of hyphen.
HA partial backup doesn't allow to use addons with hyphen and forces to update the slug to underscore. When I do that, the job finishes, but doesn't pick up any addon.  It is advised to update the slug to bring back the partial backup compatibility

The only downside of this change is that there is no option to upgrade from the previous version. So the add-on should be reinstalled and configuration should be moved to the new addon_config folder

I'll tag it as a breaking change

Fixed in core.
https://github.com/home-assistant/core/pull/115274
I'll revert

Oh wow. That's even better! Thank you for your effort


[Critical] App won't start with in absence of internet connectivity. Even if cache is enabled.

OS: MacOS Monterey 12.4
Envkey Version:  ENVKEY_SOURCE_VERSION = "2.0.22"
print("EK Should cache? " + os.environ.get("ENVKEY_SHOULD_CACHE"))
import envkey
secrets = envkey.fetch_env(cache_enabled=True)
print("..Done!")

With WiFi on, this works ok.
With WiFi off, it hangs after the first print.
A few issues with this:

Notice how my import envkey is unnaturally after the first print. I had to move it down, otherwise the import envkey hangs on network. This completely invalidates the presence of cache_enabled=True as fetch_env function argument.

Envkey should detect no network interface is up, and immediately fallback to loading from ~/.envkey/cache/xxxxx file (which in turn, is correctly updated according to filesystem timestamps).

Even if the network was up, but dropping packets, a configurable timeout should be applied before falling back to cache.

Looking into this. There's a fairly long timeout and a number of retries before falling back to cache in order to handle patchy network conditions (the timeout is configurable when using envkey-source directly but not via language libraries). It would definitely be good to revert to the cache immediately when there's no network interface.

Hey @sscarduzio I've tried this a bunch of different ways and am unable to reproduce. I'm seeing immediate results from cache with wifi off, at least on Monterey 12.1. I can try upgrading to see if there's some difference in 12.4 but it seems unlikely.
One thing to confirm: for caching to work, you have to first load an ENVKEY successfully with caching enabled (with wifi on). Perhaps you ran your example before first loading that ENVKEY successfully?

OK I got to the bottom of this: it was Tailscale.
It creates its own interface, and it stays open even when you are disconnected. The TCP stack in my machine believed it was connected because of that.
Go figure. Thanks for your investigation anyways @danenania!
Anyways, one of my initial observations remains valid: why import envkey already loads all variables? This is a blocking operation that freezes all the app, shouldn't we effectively load the variables when fetch_env is called (honoring the caching option)?

Ok, I'm glad to hear you got to the bottom of it!
I think it should still timeout eventually but it would take awhile with the high default timeouts and all the retries--so it could seem like it's just hanging. That said, if it's really just hanging forever then that's a bug that should be fixed.
I'll close this for now but if you confirm it's hanging forever under these conditions then I can re-open or make a new issue to fix that.

Anyways, one of my initial observations remains valid: why import envkey already loads all variables? This is a blocking operation that freezes all the app, shouldn't we effectively load the variables when fetch_env is called (honoring the caching option)?

For this you can set ENVKEY_DISABLE_AUTOLOAD=1 to prevent loading on import.

Hi @danenania, thanks for the hint about ENVKEY_DISABLE_AUTOLOAD, this is very helpful!
BTW I could reproduce the hang for a long time issue without depending on Tailscale, and measure how long it hangs (~4 minutes).
The long timeout happens in general when the default gateway drops packets.
To reproduce this on MacOS:

delete the default route
add a new default route to an IP that does not forward packets anywhere

$ sudo route delete default
$ sudo route add default <IP_ADDRESS>

At this point if you run this Python code:

from time import time

t1 = time()
print("loading envkey...")
try:
    import envkey
except Exception as e:
    print(str(e))
t2 = time()
print(f'⏰ loaded in {(t2 - t1):.4f} seconds')

You eventually get this error:
loading envkey...
ENVKEY invalid. Couldn't load vars.
⏰ loaded in 245.0167 seconds
envkey-source echo 'error: could not load from server.
fetch error: Get "https://api-v2-2.envkey.com/fetch?fetchServiceVersion=2&envkeyIdPart=<redacted>&clientName=envkey-source&clientVersion=2.0.22&clientOs=darwin&clientArch=arm64": context deadline exceeded (Client.Timeout exceeded while awaiting headers)'; false

Process finished with exit code 0

I hope this is helpful.

Thanks @sscarduzio that sounds about right given multiple failovers and retries with exponential backoff though obviously it does also seem too long for a dev situation--it might be reasonable on a server when there's no cache to fall back to, but it should probably fail sooner than that and revert to the cache when caching is enabled.


[ChatLLaMA] RL Trainer - is_deepspeed_init variable initialization

The variable self.is_deepspeed_init is initialized only if deepspeed is enabled in the actor or critic - line
In my case, i trained both actor and reward without enabling Deepspeed. Hence, the while training the RL model, i received error on this line.
Initializing the variable to None solves the problem.

Hi @shrinath-suresh, thanks again for the contribution!


CUDA Error

I am trying to execute the program, but I get some errors, and I can't do. I installed dependencies without errors and checkpoints
napalm@napalm-Z490-UD:~/Documentos/vqgan-clip-main$ pip install -r requirements.txt Collecting git+https://github.com/openai/CLIP.git (from -r requirements.txt (line 10))   Cloning https://github.com/openai/CLIP.git to /tmp/pip-req-build-00rpswv6   Running command git clone -q https://github.com/openai/CLIP.git /tmp/pip-req-build-00rpswv6 Requirement already satisfied (use --upgrade to upgrade): clip==1.0 from git+https://github.com/openai/CLIP.git in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 10)) Requirement already satisfied: absl-py==0.13.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 1)) (0.13.0) Requirement already satisfied: aiohttp==3.7.4.post0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 2)) (3.7.4.post0) Requirement already satisfied: antlr4-python3-runtime==4.8 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 3)) (4.8) Requirement already satisfied: async-timeout==3.0.1 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 4)) (3.0.1) Requirement already satisfied: attrs==21.2.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 5)) (21.2.0) Requirement already satisfied: backcall==0.2.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 6)) (0.2.0) Requirement already satisfied: cachetools==4.2.2 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 7)) (4.2.2) Requirement already satisfied: certifi==2021.5.30 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 8)) (2021.5.30) Requirement already satisfied: chardet==4.0.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 9)) (4.0.0) Requirement already satisfied: decorator==5.0.9 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 11)) (5.0.9) Requirement already satisfied: einops==0.3.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 12)) (0.3.0) Requirement already satisfied: fsspec==2021.6.1 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 13)) (2021.6.1) Requirement already satisfied: ftfy==6.0.3 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 14)) (6.0.3) Requirement already satisfied: future==0.18.2 in /usr/lib/python3/dist-packages (from -r requirements.txt (line 15)) (0.18.2) Requirement already satisfied: grpcio==1.38.1 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 16)) (1.38.1) Requirement already satisfied: idna==2.10 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 17)) (2.10) Requirement already satisfied: imageio==2.9.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 18)) (2.9.0) Requirement already satisfied: imageio-ffmpeg==0.4.4 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 19)) (0.4.4) Requirement already satisfied: ipython==7.25.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 20)) (7.25.0) Requirement already satisfied: ipython-genutils==0.2.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 21)) (0.2.0) Requirement already satisfied: jedi==0.18.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 22)) (0.18.0) Requirement already satisfied: kornia==0.5.4 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 23)) (0.5.4) Requirement already satisfied: Markdown==3.3.4 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 24)) (3.3.4) Requirement already satisfied: matplotlib-inline==0.1.2 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 25)) (0.1.2) Requirement already satisfied: multidict==5.1.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 26)) (5.1.0) Requirement already satisfied: numpy==1.21.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 27)) (1.21.0) Requirement already satisfied: oauthlib==3.1.1 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 28)) (3.1.1) Requirement already satisfied: omegaconf==2.1.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 29)) (2.1.0) Requirement already satisfied: packaging==20.9 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 30)) (20.9) Requirement already satisfied: parso==0.8.2 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 31)) (0.8.2) Requirement already satisfied: pexpect==4.8.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 32)) (4.8.0) Requirement already satisfied: pickleshare==0.7.5 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 33)) (0.7.5) Requirement already satisfied: Pillow==8.2.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 34)) (8.2.0) Requirement already satisfied: prompt-toolkit==3.0.19 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 35)) (3.0.19) Requirement already satisfied: protobuf==3.17.3 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 36)) (3.17.3) Requirement already satisfied: ptyprocess==0.7.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 37)) (0.7.0) Requirement already satisfied: pyasn1==0.4.8 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 38)) (0.4.8) Requirement already satisfied: pyasn1-modules==0.2.8 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 39)) (0.2.8) Requirement already satisfied: pyDeprecate==0.3.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 40)) (0.3.0) Requirement already satisfied: Pygments==2.9.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 41)) (2.9.0) Requirement already satisfied: pyparsing==2.4.7 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 42)) (2.4.7) Requirement already satisfied: pytorch-lightning==1.3.7.post0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 43)) (1.3.7.post0) Requirement already satisfied: PyYAML==5.4.1 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 44)) (5.4.1) Requirement already satisfied: regex==2021.4.4 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 45)) (2021.4.4) Requirement already satisfied: requests==2.25.1 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 46)) (2.25.1) Requirement already satisfied: requests-oauthlib==1.3.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 47)) (1.3.0) Requirement already satisfied: rsa==4.7.2 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 48)) (4.7.2) Requirement already satisfied: six==1.16.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 49)) (1.16.0) Requirement already satisfied: taming-transformers==0.0.1 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 50)) (0.0.1) Requirement already satisfied: tensorboard==2.4.1 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 51)) (2.4.1) Requirement already satisfied: tensorboard-plugin-wit==1.8.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 52)) (1.8.0) Requirement already satisfied: torch==1.9.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 53)) (1.9.0) Requirement already satisfied: torchaudio==0.9.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 54)) (0.9.0) Requirement already satisfied: torchmetrics==0.3.2 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 55)) (0.3.2) Requirement already satisfied: torch-optimizer==0.1.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 56)) (0.1.0) Requirement already satisfied: torchvision==0.10.0 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 57)) (0.10.0) Requirement already satisfied: tqdm==4.61.1 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 58)) (4.61.1) Requirement already satisfied: traitlets==5.0.5 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 59)) (5.0.5) Requirement already satisfied: typing-extensions==<IP_ADDRESS> in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 60)) (<IP_ADDRESS>) Requirement already satisfied: urllib3==1.26.6 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 61)) (1.26.6) Requirement already satisfied: wcwidth==0.2.5 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 62)) (0.2.5) Requirement already satisfied: Werkzeug==2.0.1 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 63)) (2.0.1) Requirement already satisfied: yarl==1.6.3 in /home/napalm/.local/lib/python3.8/site-packages (from -r requirements.txt (line 64)) (1.6.3) Requirement already satisfied: setuptools>=18.5 in /usr/lib/python3/dist-packages (from ipython==7.25.0->-r requirements.txt (line 20)) (45.2.0) Requirement already satisfied: google-auth<2,>=1.6.3 in /home/napalm/.local/lib/python3.8/site-packages (from tensorboard==2.4.1->-r requirements.txt (line 51)) (1.35.0) Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /home/napalm/.local/lib/python3.8/site-packages (from tensorboard==2.4.1->-r requirements.txt (line 51)) (0.4.6) Requirement already satisfied: wheel>=0.26; python_version >= "3" in /usr/lib/python3/dist-packages (from tensorboard==2.4.1->-r requirements.txt (line 51)) (0.34.2) Requirement already satisfied: pytorch-ranger>=0.1.1 in /home/napalm/.local/lib/python3.8/site-packages (from torch-optimizer==0.1.0->-r requirements.txt (line 56)) (0.1.1) Building wheels for collected packages: clip   Building wheel for clip (setup.py) ... done   Created wheel for clip: filename=clip-1.0-py3-none-any.whl size=1369091 sha256=b6fd2947ce04127529aa7039325c06c13a87b989971582c972d15cbced85e69c   Stored in directory: /tmp/pip-ephem-wheel-cache-pqn14vac/wheels/ab/4f/3a/5e51521b55997aa6f0690e095c08824219753128ce8d9969a3 Successfully built clip
But when I rund the main.py it give some errors. What it is the problem and how to solve? I am noob on Python
napalm@napalm-Z490-UD:~/Documentos/vqgan-clip-main$ python3 main.py -p "Alien World" /home/napalm/.local/lib/python3.8/site-packages/kornia/augmentation/augmentation.py:1830: DeprecationWarning: GaussianBlur is no longer maintained and will be removed from the future versions. Please use RandomGaussianBlur instead.   warnings.warn( /home/napalm/.local/lib/python3.8/site-packages/pytorch_lightning/metrics/__init__.py:43: LightningDeprecationWarning: pytorch_lightning.metrics.module has been renamed totorchmetrics.and split off to its own package (https://github.com/PyTorchLightning/metrics) since v1.3 and will be removed in v1.5   rank_zero_deprecation( /home/napalm/.local/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:572: DeprecationWarning:np.objectis a deprecated alias for the builtinobject. To silence this warning, use objectby itself. Doing this will not modify any behavior and is safe.  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations   (np.object, string), /home/napalm/.local/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:573: DeprecationWarning:np.boolis a deprecated alias for the builtinbool. To silence this warning, use boolby itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, usenp.bool_here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations   (np.bool, bool), /home/napalm/.local/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:597: DeprecationWarning:np.objectis a deprecated alias for the builtinobject. To silence this warning, use objectby itself. Doing this will not modify any behavior and is safe.  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations   types_pb2.DT_STRING: np.object, /home/napalm/.local/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:601: DeprecationWarning:np.boolis a deprecated alias for the builtinbool. To silence this warning, use boolby itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, usenp.bool_here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations   types_pb2.DT_BOOL: np.bool, /home/napalm/.local/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:618: DeprecationWarning:np.objectis a deprecated alias for the builtinobject. To silence this warning, use objectby itself. Doing this will not modify any behavior and is safe.  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations   types_pb2.DT_STRING_REF: np.object, /home/napalm/.local/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:623: DeprecationWarning:np.boolis a deprecated alias for the builtinbool. To silence this warning, use boolby itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, usenp.bool_here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations   types_pb2.DT_BOOL_REF: np.bool, /home/napalm/.local/lib/python3.8/site-packages/tensorboard/util/tensor_util.py:113: DeprecationWarning:np.objectis a deprecated alias for the builtinobject. To silence this warning, use objectby itself. Doing this will not modify any behavior and is safe.  Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations   np.object: SlowAppendObjectArrayToTensorProto, /home/napalm/.local/lib/python3.8/site-packages/tensorboard/util/tensor_util.py:114: DeprecationWarning:np.boolis a deprecated alias for the builtinbool. To silence this warning, use boolby itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, usenp.bool_here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations   np.bool: SlowAppendBoolArrayToTensorProto, Working with z of shape (1, 256, 16, 16) = 65536 dimensions. loaded pretrained LPIPS loss from taming/modules/autoencoder/lpips/vgg.pth VQLPIPSWithDiscriminator running with hinge loss. Restored from checkpoints/vqgan_imagenet_f16_16384.ckpt Traceback (most recent call last):   File "main.py", line 272, in <module>     z_min = model.quantize.embedding.weight.min(dim=0).values[None, :, None, None] RuntimeError: CUDA error: no kernel image is available for execution on the device CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
Thanks!

Are you sure that you have a CUDA-compatible version of PyTorch installed, and the correct drivers for your GPU?

I really don't know what PyTorch is, but as I understand it is installed when I do pip install -r requirements.txt, it don't give me errors.
For the nVidia drivers, it was installed automatically to latest version when I installed SO, I am on a clean install of Ubuntu, my GPU it is a RTX3080

Try reinstalling PyTorch from the website directly

pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html
pip install ftfy regex tqdm omegaconf pytorch-lightning IPython kornia imageio imageio-ffmpeg einops torch_optimizer
I tried to install PyTorch from their website, but has the same problem. I tried this two lines commands from another github notebook, and now it works... Now I have a problem with my PSU, do not have enough power for the 3080 and its shotdown, but this is another problem. So I don't fully tried but I think it will works

Hmmm…. Well, at a minimum I’m glad you got it working. I didn’t have an issue installing from source on a fresh GPU last week, so I think it must  be a GPU-specific problem. Unfortunately I don’t have access to a 3080, so I can’t really experiment with things. Hopefully this issue will be helpful to the next person though.


Docs mention but do not link to "hyper"

Where is this hyper?

Here: https://github.com/hyperium/hyper

See pull request which should trivially resolve this, for future readers.


Truncating local records

Fixes #924

I added some error messages in the text, although didn't put them in the common error messages section. Should still be discoverable through docs search.
One thing I might want to do is include Metin's example of how to reproduce the problem of local constraints on stale data.
Other than that, does this look good?


api: add file_id into jsonschema

We should add the file_id (i.e. FileInstance UUID) into the serialized record. This requires changing JSONSchema/Mapping/Marshmallow? Understand impact on existing services like CDS Videos and Zenodo of this change.

#80 only adds file_id to the JSONSchema, but it doesn't add the file_id into the JSON structure being dumped. See https://github.com/inveniosoftware/invenio-records-files/blob/master/invenio_records_files/api.py#L68-L77


fix notification pojo

make sure validate is called on notifications
add unit test
clean up the pojo a bit

I hate travis. This all passes on my box. I'm merging.


https://github.com/pyjanitor-devs/pyjanitor/blob/dev/api/functions

Brief Description of Fix
The hyperlink is broken to "General Functions" on the pyjanitor homepage
It gives a 404 error, even though the URL looks complete as "https://github.com/pyjanitor-devs/pyjanitor/blob/dev/api/functions"
Click on the link to find out where it takes you...
Quick start
Installation: conda install -c conda-forge pyjanitor
Check out the collection of [general functions](https://github.com/pyjanitor-devs/pyjanitor/blob/dev/api/functions)

Thanks @richlysakowski ... Do you mind creating a PR to fix this?


Redis port does not need to be published

Description
Is there a reason to publish redis service port ?
It seems to me that publishing it in docker-stack.yml is unecessary (redis is accessed internally in the swarm by other services).
When deploying the stack, one can see the published port and try to access it from a web browser but will have the following message which can be confusing :
-ERR wrong number of arguments for 'get' command

Steps to reproduce the issue, if relevant:

deploy the stack
open http://<host_ip>:<redis_published_port>

You'r right, redis does not need to be published as it's only used internally by the vote and worker services.
Quick note regarding your example:
Be careful, redis does not expose a http endpoint but a tcp one, if you use a redis client you will be able to connect to the redis service from the outside. Below and nxample when running the stack on play-with-docker.com:
$ redis-cli -h ip172-18-0-10-bahv0ckprd2000cias1g-30000.direct.labs.play-with-docker.com -p 30000
ip172-18-0-10-bahv0ckprd2000cias1g-30000.direct.labs.play-with-docker.com:30000>

Note: as no host port is provided in the redis service, the first one of the range (30000) is mapped to 6379.

You'r right, redis does not need to be published as it's only used internally by the vote and worker services.

I just ran into this problem also. So can anything be done here? When links popup at the top of PWD I think most people's instinct is to click on them. Seeing an error message might make people think their example hasn't been setup correctly.

During the Black Hat 2018 conference, example-voting-app was used as a demo to demonstrate some security vulnerabilities of Docker (see around 26min at https://www.youtube.com/watch?v=HTM3ZrSvp6c, for more info, see https://hornecyber.com/media/2420/def-con-26-white-paper-v3.pdf)
I just checked with the docker-compose.yml, removing the 6379 port doesn't break anything as mentioned above.
As good practice, I would change that but I did not tested all the other docker-compose and such.

Can someone provide a PR for me? There are several compose/stack files, and a kube deployment file.


2.5.4

@rultor release tag is 2.5.4

@rultor release tag is 2.5.4

@portlek OK, I will release it now. Please check the progress here

@rultor release tag is 2.5.4

@portlek Done! FYI, the full log is here (took me 8min)


Broken config

Hi! Please fix this https://github.com/kreta/Kreta/commit/20fc8eddf36e72c337a83b073cc2f66ce218584c#diff-ec47a176cac6c1160be1518eb8705a8cR23 Thanks)

Ufff... thanks for reporting the issue! :)

@benatespina deploy please :)


Support PersistentVolumeClaim retention feature

Checks

[X] I have checked for existing issues.
[X] This report is about the User-Community Airflow Helm Chart.

Motivation
Sometimes we need extra workers for large tasks, but these tasks don't occur very often.
We prefer not to keep workers on standby all the time, as this helps reduce costs.
Implementation
Ref: https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#persistentvolumeclaim-retention
Add .spec.persistentVolumeClaimRetentionPolicy property to the template file.
Are you willing & able to help?

[ ] I am able to submit a PR!
[X] I can help test the feature!

@Setsushin we don't currently mount any PVCs on the worker StatefulSet by default.
Are you mounting one for some reason, and if so, can you confirm what you use this PVC for?

@thesuperzapper
Sorry, this is a misunderstanding. I found that the chart I used is not the community version but the official version. The official version defaults to using PVC and supports the configuration ofpersistentVolumeClaimRetentionPolicy.
Thanks for the reply and have a good day.


Support for weaviate vector store

Added support for the Weaviate vector store #340

@sunilkumardash9 thank your contribution. please use the dev as the target branch

@SimFG I think we need to update the SQLite versions. Python 3.8 probably ships with an older version that isn't supported anymore in chroma.

@sunilkumardash9 it will be fixed in #488 pr

@sunilkumardash9 i will merge the pr: https://github.com/zilliztech/GPTCache/pull/493, because the pr fully meet the entry requirements.
Do you have some ideas? If have, please tell me as soon as possible, it is expected that I will merge today.

@SimFG No sir, you are free to merge.

@sunilkumardash9 thank for your contribution and attention

@sunilkumardash9 i will close the pull request. And if you have more ideas, please feel free to open a new pull request


Fix the certificate approval command

Certificate approval is done using oc adm.
Signed-off-by: Stephen Kitt<EMAIL_ADDRESS>

/cc @tpantelis @dfarrell07 @Jaanki


Lowercase $side variable for $this->db->like() in Query Builder

$this->db->like('name',$value,'AFTER') returns LIKE '%$value%'. Safer to lowercase for coding speed and developer habits.

This is a CI-specific feature, so how could you possibly have developed a habbit to write 'AFTER' instead of 'after'?

You are right, but most of developers are using various frameworks as I do. To remember all is not possible. Using uppercase in queries  may help to develop that habbit. Anyways, please easy :) It's just an advise to do slightly better ( as you all did the magnificent work already).

My point is - it's not a part of a query that you're writing here with 'after' ... The whole point of the query builder is that you don't write any part of a query anywhere, you just pass data.
Anyway, the build is failing because there's a typo in your patch.

Saw the typo now, sorry. OK then, I am closing, thanks.

Don't close it ... just push a fix for it.

OK I'll build a new pull request without typos. Thanks

Actually #3729 is without typo. Is it ok?

No ... #3729 works only with like(), leaving or_like(), not_like(), etc. unchanged. This here is the proper one, just fix the typo ...
You don't need to open a new PR either, just commit & push to the same branch.

I thinks it's done, thanks for your time and follow up.

Eh ... you also needed to reopen the PR.

Openned! No: #3739. Title: #3727 Lowercase $side variable for $this->db->like() in Query Builder

You had to reopen this one ...


Refactor project to match dataset

By utilizing SQLAlchemy Object Relationship Mapping, CRUD operations will be easier to manage across the various tables.

Adding more shared code to Entity
Using SQLAlchemy relationships to query for values in other tables for DBProject
Factoring out common code from Dataset

@Bento007 - what issue should this PR be connected to?


TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.
If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.
If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/74171

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/74328

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/74391

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/77062

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/80367

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/80366

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/81239

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/89614

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/92720

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/95571

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/106629

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/120848

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request
This extra notification is being sent because I expected a tag to exist by now, but it doesn't.
You may want to check your TagBot configuration to ensure that it's running, and if it is, check the logs to make sure that there are no errors.

Triggering TagBot for merged registry pull request: https://github.com/JuliaRegistries/General/pull/121184


Remove CF_API_FORCE_SECURE from manifest and update documentation for CF deployment

…or CF deployment

 (0.5) CF Deploy: Remove the CF_API_FORCE_SECURE env var. Check if API Endpoint is http or https in the backend


V value not handled correctly

Hi,
I noticed that when passing in a signature that has the recid as either 27,28, 29 or 30, the library crashes stating it is not the recid - 27 i.e. 0,1,2,3. At the very least the library should be able to handle both types and the default is to provide the v value as 27 and above not v - 27.
Cheers

Hello @James-Sangalli
Bindings to secp256k1 were not originally supposed to be used for anything other than transaction signing and handing different ranges of V values (0-3, 27-30, or 35+) was on behalf of transaction class itself. Due to the legacy of some old Ethereum implementation I've added a protection to have either v or v-27 (so, 0-3 range for V) upon unmarshalling a raw signature , but function that marshalles R|S|V to bytes will return V value in a range of 0-3 only.
If you have some custom scenario - please post it here for my understanding.
Sincerely, Alex

hi @shamatar
I am using a custom version here: https://github.com/alpha-wallet/web3swift.git (due to dependency conflicts)
When I use ecrecover, it will only accept v -27, was this tolerance added in a later version?

When v is 1c or 1b in hex and submitted I get back the error that v must be less than 3

You should check against master branch after 169 PR

From: James Sangalli<EMAIL_ADDRESS>Sent: Thursday, June 28, 2018 10:42:31 AM
To: BANKEX/web3swift
Cc: Alexander; Mention
Subject: Re: [BANKEX/web3swift] V value not handled correctly (#167)
When v is 1c or 1b in hex and submitted I get back the error that v must be less than 3
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/BANKEX/web3swift/issues/167#issuecomment-400959431, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGKv92t8X1QE-XGKlnk9bAYaG1TIcJk3ks5uBJb3gaJpZM4U5Pxs.


Update runtime yamls

Aligning our caikit+tgis runtime yaml with https://github.com/opendatahub-io/caikit-tgis-serving/blob/main/demo/kserve/custom-manifests/caikit/caikit-tgis/caikit-tgis-servingruntime-grpc.yaml
UPDATE: there will be a slight additional update to the manifest https://github.com/opendatahub-io/caikit-tgis-serving/pull/217. So I would wait merging this test PR until we include the new changes

PR validation: rhods-ci-pr-test/2422

you need to update the link in description properly :)

PR Validation after image URL changes:

Sanity run for Caikit+TGIS and TGIS standalone: rhods-ci-pr-test/2467


Add an initial version of Bert, running on single client mode.

Importing code from @qlzh727's colab.

Thanks! Squash and merge now.


add info about the Slack channel

There's now a slack for COG's. We should add that to the site so people know about it.

URL to join slack is: https://cogeotiff.slack.com/join/shared_invite/zt-i37rlzlz-L0KYH3SWsIwzM~dbBbjM1w#/

And a short version: http://bit.ly/cog_slack

Does the channel still work? Anytime I try to launch it, it never shows anything

you can try this link: https://join.slack.com/t/cogeotiff/shared_invite/zt-17by75auf-39YWJEM08MkwEyM67TqeDg

It says "Please contact your orgs owner to resent it"

Hi. I think its not available for the public.


What is the replacement of Interceptors?

Now, there are no interceptors. How can I globally format result before sending to client?

just using regular middlewares. Example can be found here.

just using regular middlewares. Example can be found here.

@pleerock I don't think you can, the express driver sends the response before 'after' middleware said get called so currently there are no way to format response returned from @Get, @Post, etc handlers.

@pleerock the interceptor is good, why you remove it?

@pleerock @19majkel94
I have read this discussion #110 .
I don't really know about koa. Maybe it's useless in koa, but it very useful in express which without overriding method of response can't intercept the returning results.
I don't consider interceptor as middleware. I'd it to be a result processor before sending to the client.
Before, I write an interceptor to format my result. Now I need to override the method which I don't want to care about.

Interceptors are very useful. Not sure why you would want to remove them?
Yes, it is maybe possible to achieve the same functionality by using the solution, but this does not fit well with routing controllers class style. Why would I want to change res,.send() instead of having an elegant solution like @InterceptorGlobal()?
Interceptors are awesome, please do not kill them, or at least give us a good reason why you have to remove them.

@pleerock without interceptors is there a way to achieve the same functionality using classes/services and DI (typedi)?

You don't need an integration inside framework, just use the language features 😉
https://github.com/pleerock/routing-controllers/issues/110#issuecomment-302504119

@19majkel94 this works on a controller level, but with this solution you cannot implement global interceptor applied to all application endpoints.

Yeah, but what's the problem with one-line decorator for controller?
Maybe let's @pleerock talk, we've decided to remove this feature as the sample use-case wasn't enough to convince that it's a good feature and it seemed that only a few people used it.
But formating REST responses to the unified format is a good use-case, I would also like to see and error interceptor for the same reason.

@19majkel94, the problem is because some developer on a project can forget to put that controller. Also, following "separation of concerns", there are usually more than few interceptors. Having an option to decorate classes/components as global application wide interceptors is (was 😢 ) a good feature. And I assure you it was used - not sure how you came to a conclusion that no one is using it.

Right some people are using this functionality, some do not. Its probably was a mistake to remove this functionality because sometimes it can be really useful for users. And as we can see users are using this functionality. Returned it back and will be released in 0.7.0-alpha.10


Removing nonsensical asserts

This PR removes asserts that are not helpful.

Why do I get smaller code coverage after I remove lines?
I'll add a test.

The new test doesn't hit what I wanted it to.

I'll follow both advice in this PR.  Neat.


Fix ranges for Brazilian zip codes

Description
This fixes the issue #1123

Locale updates/fixes 💓💓 💟💟
@lucasqueiroz  Thanks for contributing 🥇


Update

How to update to the latest commonmark?

You can use the latest dev version:
composer req webuni/commonmark-table-extension:@dev league/commonmark:@dev


Stub tests

Stub tests with OHHTTPStubs.
Instead of connecting directly to the service, we can pass through the OHHTTPStubs library. Depending on the state of a flag, we will either (a) connect to the service, save the response to a file, and return the response to the requesting class, or (b) load a saved response from file and return it to the requesting class.
This way, we get the best of both worlds. By programmatically storing responses (and updating them with each release) we can make sure to stay consistent with the services as they change. But with stubbed responses, we can also make our tests run quickly and consistently.

When saving the response to a file, we might be able to use reflection to get the name of the current test method. That name will be helpful when reading/writing the stubbed response to a file.
We should also investigate a post-action script for each scheme that copies the files from the simulator to the project directory. See this page with variables accessible from Xcode.

Stephanie made a great comment in #407 about Capital One's SWHttpTrafficRecorder.

Closing as will be addressed in upcoming changes to unit test generation


can we deploy to vercel (with nitro) ?

hi,
ive been trying to use your plugin to create a fastify based api (compatible with nitro) and then deploy it to vercel.
vercel supports the prebuilt option (https://vercel.com/guides/deploying-locally-built-nextjs) using vercel deploy --prebuilt
could u let me know if this works ? i tried a few things, but i was not able to get ur plugin and fastify to bundle to the .vercel folder

Sorry for the late response, but I don't use Vercel personally and I stopped maintaining it for a while due to some inconsistencies with Nitro when I first did this.


Adding video in About Us section

Adding video in About Us  section about working of website and how easily users can shop on Vigybag will improve user experience and users can easily make use of website.

You have any video regarding this ?

Yes, I have a video


Update dd-plist to 1.25

Updates com.googlecode.plist:dd-plist from 1.23 to 1.25.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Adjust future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.googlecode.plist", artifactId = "dd-plist" } ]

Or, add this to slow down future updates of this dependency:
dependencyOverrides = [{
  pullRequests = { frequency = "@monthly" },
  dependency = { groupId = "com.googlecode.plist", artifactId = "dd-plist" }
}]

labels: library-update, commit-count:1

Superseded by #87.


⚠ ClamAV has degraded performance

In a0bd9e0, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 622 ms

Resolved: ClamAV performance has improved in 4b886d2.


error: RPC failed; result=22, HTTP code = 500

Trying to push about 5gb of files and cannot get pass this error:
git -c diff.mnemonicprefix=false -c core.quotepath=false commit -q -F C:\Users\David\AppData\Local\Temp\fmxamqvy.1bt
git -c diff.mnemonicprefix=false -c core.quotepath=false push -v --tags --set-upstream origin master:master
POST git-receive-pack (chunked)
fatal: The remote end hung up unexpectedly
fatal: The remote end hung up unexpectedly
error: RPC failed; result=22, HTTP code = 500
Pushing to https://git.XYZ.com:3000/XYZ.com/ZYZ.git
Everything up-to-date
Completed with errors, see above.

I am having the same issue with a push of a repository wich contains multiple files wich total size of around 3.5GB

Thank you both!
Since it's a 500 error, please post the error from Gogs...

Also, I think it is related/same to https://github.com/gogits/gogs/issues/1530

This is from http.log
2015/09/24 12:40:52 [...routers/repo/http.go:344 serviceRpc()] [E] fail to serve RPC(receive-pack): exit status 128
gogs.log have usual log
2015/09/24 16:43:42 [I] Log Mode: File(Trace)
2015/09/24 16:43:42 [I] Redis Supported
2015/09/24 16:43:42 [I] Memcache Supported
2015/09/24 16:43:42 [I] Cache Service Enabled
2015/09/24 16:43:42 [I] Session Service Enabled
2015/09/24 16:43:42 [T] Doing: CheckRepoStats
2015/09/24 16:43:42 [I] SQLite3 Supported
2015/09/24 16:43:42 [I] Run Mode: Development
2015/09/24 16:43:42 [I] Listen: http://<IP_ADDRESS>:3000
I have tried changing the log level, production or development but the log detail is doesn't change.

Can you check with this see if it helps? http://stackoverflow.com/questions/10790232/hanging-at-post-git-receive-pack-chunked

I have found that post. I tried changing that to 512MB and 1GB and I still got the same issue.

2015/09/24 12:40:52 [...routers/repo/http.go:344 serviceRpc()] [E] fail to serve RPC(receive-pack): exit status 128

From what I understand based on the code, Git has failed itself and prints all the error message to you(the client side), Gogs is transparent here.
Are you also using Nginx? The HTTP buffer can be limited by Nginx config as well.

Nope. I am not using any proxy at all. I am testing changing the buffer to 5kbytes and I will post the results here.

Still the same error.
Counting objects: 106486, done.
Delta compression using up to 4 threads.
Compressing objects: 100% (39200/39200), done.
Writing objects: 100% (106486/106486), 2.59 GiB | 11.19 MiB/s, done.
Total 106486 (delta 60867), reused 101269 (delta 55947)
error: RPC failed; result=22, HTTP code = 500
fatal: The remote end hung up unexpectedly
fatal: The remote end hung up unexpectedly
Everything up-to-date
This are the results of the push.

@carlosfvp Just a suggestion, why don't you push through SSH for this large amount of content?

Thank you for the suggestion. I will test this and I will post the results.

:wink:

How would one do this on a Windows 2012 server?
On Thu, Sep 24, 2015 at 6:20 PM, 无闻<EMAIL_ADDRESS>wrote:

[image: :wink:]
—
Reply to this email directly or view it on GitHub
https://github.com/gogits/gogs/issues/1682#issuecomment-143087172.

@FyinInc You can do that by installing ssh as any other service under Windows. At the end I figured that the server was running out of memory. I was confident this wasn't the issue because I had 8gb of ram free. There must be some kind of memory leak. As I saw on the related issue, this memory leak might not be fixed yet. Anyway, I will test tomorrow adding some extra RAM on the server.

Chunked transfer caused me some troubles when hosting apps via an HTTP server.

Maybe the http library used for gogs is not fully compatible with chunked transfers.

@carlosfvp about memory leak:
Even if there is no memory leak, Go GC can't collect objects back because it is in use while you're still transfer data.

I needed 10GB free RAM. So the HTTP library seems to work fine but at the end of the process there is still 4GB of RAM used by gogs.exe process.

Thank you!

Simply mirroring a large repository would cause a bad gateway error, it seems like Gogs is doing synchronous action to pull the target git repo after the form is submitted.

@carlosfvp thanks your info!
Which version of Gogs are you using?

Increasing the virtual memory I got:
git -c diff.mnemonicprefix=false -c core.quotepath=false push -v --tags
origin master:master
POST git-receive-pack (chunked)
fatal: The remote end hung up unexpectedly
fatal: The remote end hung up unexpectedly
error: RPC failed; result=22, HTTP code = 500
Pushing to https://git...
Everything up-to-date
On Sat, Sep 26, 2015 at 11:45 AM, 无闻<EMAIL_ADDRESS>wrote:

@carlosfvp https://github.com/carlosfvp thanks your info!
Which version of Gogs are you using?
—
Reply to this email directly or view it on GitHub
https://github.com/gogits/gogs/issues/1682#issuecomment-143475205.

Why do people not reading docs and not response to my question....?
Please read https://github.com/gogits/gogs/blob/master/CONTRIBUTING.md#bug-report :sweat_smile:

Close for now but open for continuing discussion.

I have the same problem

@Eng3l can you update to latest develop branch see if error prints more accurately?


add core as one of candidate username for ssh

Can you please add core as in coreos default user for ssh.

@simcap Looks good, everything works as expected. Thanks


Add file version string into STARTUP.DSN

Add global.design.version = "1.2" into STARTUP.DSN

Merged into master, closing


Scheduling Updates

This is an epic issue that groups scheduling updates issues based on user feedback for 1.1.0.

See https://github.com/NASA-AMMOS/aerie/milestone/3


Capitalize "IMPORT" in example query

Using lowercase "import" will give zero results since the value of the operation field is capitalized in rows written by the script.

@googlebot I signed it!


fix(kubeserver): client-go panic when cluster-deleted

3.9

FIX: https://github.com/yunionio/cloudpods/issues/16188


Remove the renderForms parameter from the Annotation getOperatorList methods

The renderForms parameter pre-dates the introduction of the general intent parameter, which means that we're now effectively passing the same state twice to these getOperatorList methods.

/botio test

Nice find!


Develpoment Testing

I have tried the development branch and i get errors all over the place.
The main index page shows this error:
Warning: filemtime(): stat failed for docs\05_More_Examples\??????_???.md in phar://D:/xampp/htdocs/channelseeder/docs/daux.phar/libs/Tree/Builder.php on line 76

And once entering the documentation there is a critical error:
Fatal error: Uncaught exception 'Todaymade\Daux\Exception' with message 'Could not locate file 'For_Developers/Creating_a_Processor'' in phar://D:/xampp/htdocs/channelseeder/docs/daux.phar/libs/Format/Base/ContentTypes/Markdown/LinkRenderer.php:41 Stack trace: #0 phar://D:/xampp/htdocs/channelseeder/docs/daux.phar/libs/Format/Base/ContentTypes/Markdown/LinkRenderer.php(67): Todaymade\Daux\Format\Base\ContentTypes\Markdown\LinkRenderer->resolveInternalFile('For_Developers/...') #1 phar://D:/xampp/htdocs/channelseeder/docs/daux.phar/vendor/league/commonmark/src/HtmlRenderer.php(80): Todaymade\Daux\Format\Base\ContentTypes\Markdown\LinkRenderer->render(Object(League\CommonMark\Inline\Element\Link), Object(League\CommonMark\HtmlRenderer)) #2 phar://D:/xampp/htdocs/channelseeder/docs/daux.phar/vendor/league/commonmark/src/HtmlRenderer.php(92): League\CommonMark\HtmlRenderer->renderInline(Object(League\CommonMark\Inline\Element\Link)) #3 phar://D:/xampp/htdocs/channelseeder/docs/daux.phar/vendor/league/commonmark/src/Block/Rend in phar://D:/xampp/htdocs/channelseeder/docs/daux.phar/libs/Format/Base/ContentTypes/Markdown/LinkRenderer.php on line 41

From what i can see it is looking in all the wrong folders. And to change this i would have to edit some obscure files hidden somewhere.
I have been trying the development branch as i have been having errors with landing pages and index pages in the master branch and i have been seeing comments saying

Can you please try the latest developmentbranch ? your problem should be fixed in there.

onigoetz

From the path, I see you are on windows, I need to do some tests on windows, as there are other reported problems. I cannot guarantee it works for the moment. Do you have a linux/ os x machine ?

Ahh i see, didnt even consider that windows file systems would be playing up with it. Ofcourse it would, had a similar problem when i was doing a directory manager for a photo gallery.
I will test it on my Linux server later today and let you know the result. Thanks for the quick response.

Hi, did you have time to check on your linux server ?

Hi,
one of the two issues you reported is fixed, the other one has no known solution yet:

The fatal exception is fixed, the page is generated correctly
The filemtime error is still present, it is because the filename is utf8 and it is not supported on windows yet. We are still investigating on what might be causing this

Closing this as it seems there is no reliable way to support this on windows, instead use http://daux.io/Features/Front_Matter to set the title of your page and a filename that doesn't contain utf8 characters


luci-mod-network: fix src/dest textvalue for rules

Fix the incorrectly displayed src/dest textvalue for the rules.
Signed-off-by: Vladislav Grigoryev<EMAIL_ADDRESS>

@jow-, is there anything else required on my part?


Write up and implementation of proposed deployment process for Dags

Write up a proposed process for deploying dag changes to MWAA

Closed with https://github.com/ucldc/rikolti/pull/472 ?


[Bug]: README CI Badge

What happened?
Link from README CI badge just leads to the badge image.  Ideally, this should lead to the actions page for the tests.
Relevant log output
No response

@scivision I'm going to let you fix this because I'm a total novice at GitHub actions...


⚠ Bofrost.at has degraded performance

In b6f464f, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 826 ms

Resolved: Bofrost.at performance has improved in 408cf5b after 7 minutes.


pyqt6-webengine: init

Description of changes
Also provides a build of qutebrowser which uses qt6.
I have come across an issue with qutebrowser consistently crashing in certain situations, so I'm gonna leave this as a draft unless/until I can get to the bottom of it. 7623600

Note: Webengine is a very heavy package to build, reviewers may wish to use my cache which already contains the build from this PR. My cache priority is set lower than the official cache.nixos.org, so once the official cache is popullated with this build, it should ignore my cache automatically:
url: https://cache.nrdxp.dev
key: cache.nrdxp.dev:KxAfP+PpOl/8UffuH1+lNr35DEsb/irKl7K7C2igbHQ=

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Thanks to upstream for pointing me to a patch that will fix the github crashes in webengine.
https://codereview.qt-project.org/c/qt/qtwebengine/+/436316
I'll apply and push a fix when the build finishes I can confirm it works.

Why was this merged so fast without proper review and without the commit messages being properly formatted?


An unexpected space in Line 191

ERROR:
File ".\InsertPeIntoPng.py", line 194
try:
^
IndentationError: unindent does not match any outer indentation level
FIX: An unexpected space in Line 191.Please fix it,thank u~

Noticed it, idk why didn't make a pr earlier busy af probably ;D https://github.com/Maldev-Academy/ExecutePeFromPngViaLNK/pull/2


Investigate why AOVs don't render correctly

This is a strange bug, in both interactive and batch mode it does render the beauty pass fine, but the other AOV passes tend to glitch out.

Working in https://github.com/boberfly/GafferCycles/commit/7163d816a68dc08a0d6f3562a87c55130d3b4ef2
Turns out the 'Film' node in addition to the buffer params just needs to be tagged to update so it knows those passes are enabled, too easy! Perhaps denoising will work also now.


Suggestions for Pop-up Commands

Unable to remove pop-up box of Suggestions for possible commands. Suggestions box will disappear if user either hits Esc key or begins typing. However, clicking the textbox will open this window that serve more as a blocking overlay over both the most recent chat history and the input text box. We wanted to provide feedback for potentially removing this feature or enabling/disabling this Suggestions box from displaying every time.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 19ddf42e-0a47-7717-52d4-e549155480a2
Version Independent ID: 4bbe9beb-233f-cfd5-097b-f280aab5fde8
Content: Microsoft Teams developer community help and feedback - Teams
Content Source: msteams-platform/feedback.md
Product: msteams
GitHub Login: @surbhigupta
Microsoft Alias: lajanuar

Thanks for your suggestion. Please raise UserVoice for the same.
User Voice URL:
https://microsoftteams.uservoice.com/forums/555103-public/filters/new


the generated primes for bit-lengths 2 .. 15 are all identical

(in bash) try:
=========================== begin bash code ==============================
i=2
while [[ "35879" == $(openssl prime -generate -bits $i) ]] ; do echo $i; ((++i)); done
===========================end bash code ==============================
Is this intended?
It tastes like a bug.
I notice that for bit-lengths >= 16, multiple calls get different primes

Yes it's known behavior.  The prime command stops at the first prime it finds.  There are only so many small-bit primes. :)
Which version of openssl are you running?  If you're not running the latest master, then there is no good way to really seed the RNG, nore to save/restore it's state across invocations of the "prime" command.
Closing this.

It's version 1.0.2, FYI

Then there is nothing that can be done.  The RNG seeding is not very good, and the number of bits requested is pretty small, so it is not surprising that you often get the same number.

@richsalz, I think you're going down the wrong path here...  RNG quality has nothing to do with this, BN_rand is perfectly capable of delivering a number with the required amount of bits.  Now, if you ask for 2 bits, the possible numbers are 0, 1, 2 and 3, so there's not much to choose randomly from.  So it's a bit absurd to get 35879 when you ask for a 2-bit prime.
(I did a quick debug session, and sure enough, BN_rand did deliver a 3, and yet, the result was 35879....  this happens consistently for all amounts of bits below 16)
Note that probable_prime in 1.0.2 and probable_prime in 1.1.0 are quite different...  as I recall, there was some development revolving very small primes in the latter, and it seems that was never backported.

Maybe I misread.  Want to re-open this?

Looks like this was discovered before and fixed by commit 96a4c31 but it was never backported to the 1.0.2 branch which has the same problem. PR #4670 does the backport.


Wrong |CHART_RELEASE| in the documentation, there is no 1.15.3

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
A bug happened! in the documentation there is not a 1.15.3 |CHART_RELEASE|

https://docs.cilium.io/en/stable/installation/k8s-install-helm/#install-cilium > k3s tab for exemple but all documentation in Cilium website > https://helm.cilium.io/

Cilium Version
1.15.3
Kernel Version
6.5.0-26-generic
Kubernetes Version
1.29
Regression
No response
Sysdump
No response
Relevant log output
No response
Anything else?
No response
Cilium Users Document

[X] Are you a user of Cilium? Please add yourself to the Users doc

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Close this issue due to when I push it, the helm chart does not exist and now yes


VnV: NFRs consistency?

In the VnV plan section 3.2 (objectives), the most important quality / NFR is said to be "Correctness".
Your SRS lists the following nonfunctional requirements: Accuracy, Usability, Maintainability and Portability.
However in the VnV plan - section 5.2, the only NFR tested is Reliability?
Should correctness be listed in 5.2 or at least in the SRS? Or reliability?

Updated section 3.2. For NFR, Reliability and portability will be tested. we will not test maintainability since we analyzed likely changes. Thanks!


ISPN-6547 Not all classes should be added to bean manager

https://issues.jboss.org/browse/ISPN-6547

Sounds good, integrating.

Pushed, thanks Sebastian!


Sorting Techniques [GSSOC'21]

Bubble Sort:-
The simplest sorting algorithm, that works on repeatedly swapping the adjacent elements if in wrong order.
Selection Sort:-
The algorithm repeatedly find the minimum element of the unsorted part and put in the beginning. There are always two part in selection sort algorithm:-

The sorted subarray;
The  unsorted subarray.

Insertion Sort:-
In insertion sort algorithm , the element from the unsorted array is inserted to its correct position in the sorted array.
Bubble Sort :-
Time complexity: O(n*n)
Space complexity: O(1)
Selection Sort:
Time complexity: O(n*n)
Space complexity: O(1)
Insertion Sort:
Time complexity: O(n*n)
Space complexity: O(1)
Language: Python.
Please assign me this issue.
@tarun26091999, @rudrakshi99
Thank you.

Hello , @tarun26091999 @rudrakshi99 @plazzy99 I want to contribute to this issue .
I will write all the codes in JAVA.
Please do assign me this issue.
Thanks.

all sorting techniques are already being worked upon.

Non till now are added yet. if you allow me I can work upon them.


Fix overly broad Angular detection

Was triggered on "rectangular[several KiB of minified script].js"
Now requires a word boundary before "angular" and allows max. 32
characters between that and ".js"

Ok, so Travis fails but it doesn't seem related to my change at all..?
By the way, the warning about the unescaped "." in my regex can be ignored. It is indeed on purpose.

Thanks!


<'aiohttp.client_exceptions.ClientPayloadError'> Response payload is not completed:  FILENAME error #xxx

Hi, first of all, thanks for this awesome script, when I tried at first it was working perfect with multiple tags without specified artist, but when I tried again on certain artist/model without tags included, it return with <'aiohttp.client_exceptions.ClientPayloadError'> Response payload is not completed: with bunch of error number after the filename. Any idea what did I wrong?

This isn't a problem you could cause by script misuse.
The <'aiohttp.client_exceptions.ClientPayloadError'> Response payload is not completed error is a per-file error, caused generally by connection getting closed/reset during download/write process (website could just go down). Just to clarify: search was successful, for this file download link was fetched and download process has started.
This may happen sometimes due to a bunch of different reasons, normally RV should just retry the file.
If this problem persists and you are unable to download using the script but downloading from the website works, provide a full cmdline and console output with --log-level trace so it gets fixed.


Adjust pylint configuration after new version

Codecov Report

Merging #413 (1d6b75b) into develop (4186a2a) will not change coverage.
The diff coverage is 85.71%.

@@           Coverage Diff            @@
##           develop     #413   +/-   ##
========================================
  Coverage    83.05%   83.05%
========================================
  Files           56       56
  Lines         5282     5282
========================================
  Hits          4387     4387
  Misses         895      895

Impacted Files
Coverage Δ

core/eolearn/core/eodata_merge.py
97.12% <ø> (ø)

features/eolearn/features/interpolation.py
85.41% <0.00%> (ø)

core/eolearn/core/core_tasks.py
98.14% <100.00%> (ø)

core/eolearn/core/eodata.py
90.54% <100.00%> (ø)

mask/eolearn/mask/utils.py
92.10% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0382dc4...1d6b75b. Read the comment docs.