Python-action: detect which dependencies changed and which targets are missing

Essentially need to get the same information in a python-action as provided by doit info on the console.
I need this to process only those dependencies of the task which have changed.
What I've tried:
def my_action(task):
    print(task.dep_changed)

However this will print all the dependencies, no matter what.
I guess it is related to this comment in the code:
        # dep_changed is set on get_status()
        # Some commands (like `clean` also uses expand_args but do not
        # uses get_status, so `changed` is not available.

And need the missing targets as well, those seem to be separate from dep_changed in doit info.

Sounds like a bug. Can you show me a minimal example to reproduce the problem?

Turns out to be a bug indeed. dep_changed is correct if one of the dependencies changed, but it is incorrect if one of the targets have been removed:
import os
import shutil

def task_copy():
    return {
        'file_dep': ['a.txt', 'b.txt'],
        'targets': ['out/a.txt', 'out/b.txt'],
        'actions': [copy_action],
    }

def copy_action(task, dependencies):
    print(task.dep_changed)
    if not os.path.exists('out'):
        os.mkdir('out')

    for dep in dependencies:
        shutil.copyfile(dep, 'out/{}'.format(dep))

Create a.txt and b.txt
Run doit
Delete out/a.txt
Run doit -v 2
Prints ['a.txt', 'b.txt'] (incorrect)
Expected ['a.txt'] because in fact b.txt is still up to date

You could also add an example of task.dep_changed to the documentation, because I had to figure it out from the source.

you can easily check witch targets are missing with python code

Ok I accept this solution.

Actually the right way to use this feature is with a parameter named changed
def my_action(changed):
    print(changed)

The current behaviour is expected. When a task is not up-to-date for any reason other than dependencies changed, it will report as if all file_dep were modified.
This was a questionable decision... do you have a specific use-case, I might reconsider this behaviour...
Also this behaviour is described on source code but not on user's docs. So definitely some action is required. https://github.com/pydoit/doit/blob/master/doit/dependency.py#L571

Actually the right way to use this feature is with a parameter named changed

Thank you, from the docs I had no idea you can inject changed

do you have a specific use-case, I might reconsider this behaviour...

My use case:

In the example above, suppose that copy_action takes a long time for each file
Because of 1) I want to copy only the files that are a) not up to date or b) their target doesn't exist
Iterating on changed sounds like a great solution, but sometimes this behavior will make the task do more work than it should

Proposed solution: if changed reported empty list in this case, I can iterate all the targets and check if any of them are missing, then add their dependency to my list of changed dependencies.

from the docs I had no idea you can inject changed

It is there:
http://pydoit.org/tasks.html#keywords-with-task-metadata

My use case:

For your use case it is better to have a separate task for each file being copied, since they are independent...
There is a cost on creating the list of changed dependencies, and this is seldom used in practice.
This is an old feature, today i would implement this feature in a different way... anyway for now I am more inclined to set changed to None, meaning it was not calculated.

For your use case it is better to have a separate task for each file being copied, since they are independent...

The number of files and their names are not known in advance in the use case. As far as I know doit needs a method for each task. In your suggestion one would need to generate methods (one task for one file), how would you do that?

I am more inclined to set changed to None, meaning it was not calculated

I understand there is no way to know which target depends on which dependency, because the description is general.
I suggest you could introduce dependency/target description like "folder/dep*.txt" and "folder/target*.txt" meaning a 1:1 target:dependency relationship. GNU make has a similar feature, eg.:
out/%.png: input/%.png

With these restrictions it would be possible to calculate the changed variable, which is essential for supporting incremental build properly.

Sorry, I do not agree with Make style regular expressions to define dependencies and targets.
If you do not know the name of files in advance you could use delayed-tasks.
Here is an example:
http://pydoit.org/task_creation.html#delayed-task-creation

I do not agree with Make style regular expressions to define dependencies and targets
Why? It would be a lot easier to use than the delayed task creation you linked.

@patrikturi concerning your last comment about generic expressions, I think that have the same use case and I solved it easily by creating one subtask per file dynamically.
In my case I have a bunch of data download definition file (.ddl) and for each of them I want to create a csv file .csv :
DataInfo = namedtuple('DataInfo', ('name', 'ddl_file', 'csv_file'))
"""Represents information about a dataset. Contains a name, a download definition file and an actual csv data file"""

def list_all_data_to_download(data_defs_folder):
    """ Lists all datasets from the definition files in the folder"""
    return [DataInfo(f.stem, f, f.with_suffix('.csv'))
            for f in data_defs_folder.glob('**/*.ddl')]

ALL_DATA = list_all_data_to_download('./data/raw')

def task_download_data():
    """
    Downloads data file `./data/raw/**/<dataset>.csv` for each definition file `./data/raw/**/<dataset>.ddl`.
    """
    for data in ALL_DATA:
        yield {
            'name': data.name,
            'file_dep': [data.ddl_file, DATA_DDL_PYSCRIPT],
            'actions': ["python %s --ddl_csv %s" % (DATA_DDL_PYSCRIPT, data.ddl_file)],
            'verbosity': 2,
            'targets': [data.csv_file]
        }

Of course the part at the beginning (DataInfo class and list_all_data_to_download function) could very much be eased by providing a helper function (in tools.py). That helper function would be responsible to provide the pattern-related experience, but that would not leak into the main doit design: @schettino72 you will therefore probably be ok with that ?
The example above could become:
from doit.tools import list_items

ALL_DATA = list_items('DataInfo',
                      from=dict(ddl_file='./data/raw/**/*.ddl'),
                      to=dict(csv_file='./data/raw/**/*.csv'),
                      # name=lambda p: p.stem  that would not be needed (default behaviour)
                      )

def task_download_data():
    # task completely unchanged - see above.

Or whatever better api we might think of for list_items (I drafted that in 1 minute, it is probably not the best we can do)
Would something like that satisfy both the user (@patrikturi ) and the maintainer ( @schettino72 ) ?

Also concerning the other topic in this ticket: I had the issue today to try to debug "why did a task run again ??? It should not".
Being able to print to stdout the reason why tasks are run (dependency change or file dep change or target change) using a simple commandline flag (for example a maven-style flag doit -X or a pytest-style doit -v) would really be a killer feature - if it is not already there (in which case it should probably be highlighted in the doc in a better way). Should we open a ticket for this in particular ?

@smarie Thank you! This helper is exactly what I needed.

You're welcome @patrikturi !


configurable batch size

Coverage increased (+0.01%) to 45.636% when pulling 14ecbff30f26f016622562318aa8a870d9634432 on batch-size into 5556c2582c71e9c0fd9526d46fec50eae6c71730 on DAME-Aug-Sept-sprint-staging.


Update ballerina version

Purpose

$subject

Codecov Report

Merging #295 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #295   +/-   ##
=========================================
  Coverage     62.15%   62.15%
  Complexity      124      124
=========================================
  Files            13       13
  Lines           650      650
  Branches         70       70
=========================================
  Hits            404      404
  Misses          209      209
  Partials         37       37

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ead678c...9391a37. Read the comment docs.


[Request]: do you need maintainers?

Expected behavior
The repo is regularly updated.
Current behavior
Issues and PRs seem to be piling up.
Possible solution
No response
Context
No response
Notes
No response

@robertoschiavone We are always welcoming new contributors! We could probably use someone that would be interested in churning through the backlog. For what it's worth, I think the current team is more active during northern winter months.

It is within the northern winter months now but I don't think the current team is more active.


python310Packages.snowflake-connector-python: 2.9.0 -> 3.0.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python310Packages.snowflake-connector-python is: Snowflake Connector for Python
meta.homepage for python310Packages.snowflake-connector-python is: https://github.com/snowflakedb/snowflake-connector-python
meta.changelog for python310Packages.snowflake-connector-python is: https://github.com/snowflakedb/snowflake-connector-python/blob/v3.0.0/DESCRIPTION.md
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
The tests defined in passthru.tests, if any, passed
found 3.0.0 with grep in /nix/store/dwmhkbyvy2vp0ds6xjdk9xd1gw4yxxq5-python3.10-snowflake-connector-python-3.0.0
found 3.0.0 in filename of file in /nix/store/dwmhkbyvy2vp0ds6xjdk9xd1gw4yxxq5-python3.10-snowflake-connector-python-3.0.0

Rebuild report (if merged into master) (click to expand)

4 total rebuild path(s)

4 package rebuild(s)

First fifty rebuilds by attrpath
python310Packages.snowflake-connector-python
python310Packages.snowflake-sqlalchemy
python311Packages.snowflake-connector-python
python311Packages.snowflake-sqlalchemy

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/dwmhkbyvy2vp0ds6xjdk9xd1gw4yxxq5-python3.10-snowflake-connector-python-3.0.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python310Packages.snowflake-connector-python https://github.com/r-ryantm/nixpkgs/archive/f87e3de07d62c5811a6392141110f2139bd4b8b0.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/dwmhkbyvy2vp0ds6xjdk9xd1gw4yxxq5-python3.10-snowflake-connector-python-3.0.0
ls -la /nix/store/dwmhkbyvy2vp0ds6xjdk9xd1gw4yxxq5-python3.10-snowflake-connector-python-3.0.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

2 packages failed to build:

python310Packages.snowflake-sqlalchemy
python311Packages.snowflake-sqlalchemy

2 packages built:

python310Packages.snowflake-connector-python
python311Packages.snowflake-connector-python

Maintainer pings

@ofborg build python311Packages.snowflake-sqlalchemy

Requirement already satisfied: sqlalchemy<2.0.0,>=1.4.0 in /nix/store/22zyg56741pzh40mly5civwnyq74x2j4-python3.11-SQLAlchemy-1.4.46/lib/python3.11/site-packages (from snowflake-sqlalchemy==1.4.5) (1.4.46)
ERROR: Could not find a version that satisfies the requirement snowflake-connector-python<3.0.0 (from snowflake-sqlalchemy) (from versions: none)
ERROR: No matching distribution found for snowflake-connector-python<3.0.0

@ofborg build python311Packages.snowflake-sqlalchemy


[Bug]: The field order of the constraint may be different from that of the ddl

ODC version

OB version

What happened?
The field order of the constraint may be different from that of the ddl
What did you expect to happen?
The field order of the constraint is same to the ddl
How can we reproduce it (as minimally and precisely as possible)?
Create a multi-field constraint
Anything else we need to know?
No response
Cloud
No response

fixed :https://github.com/oceanbase/odc/pull/1948

pass


Data Connection couldn't be found on this project. Can I get database file .mdf?

mr. @chsakell I can't run this project because mdf file not found in data connection. Could you send me?

read the post or follow the installation instructions.

Instruction
3.change the connection string inside the HomeCinema.Data/App.config and HomeCinema.Web.config according to your development environment.
How to create database match to entities because i dont know database structure


Fix: Remove climetlab to be replaced by earthkit

Replaces climetlab with earthkit
Added to CHANGELOG
Adjusted pyproject.toml

Closes #7

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@gmertes At the moment, yes it is not backwards compatible. It might be possible to make it so

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 85.50%. Comparing base (48f21cd) to head (d264d45).
Report is 3 commits behind head on develop.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop       #8   +/-   ##
========================================
  Coverage    85.50%   85.50%
========================================
  Files            2        2
  Lines           69       69
========================================
  Hits            59       59
  Misses          10       10

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Worker cleanup (WIP)

Have more to do (want to merge ExternalTaskProcess and TaskProcess etc).
Keeping an open PR to be able to keep an eye on Travis and see if people have strong opinions against anything I'm doing

Will squash once this is done


Fix \n in Zulip message

https://rust-lang.zulipchat.com/#narrow/stream/269128-miri/topic/Cron.20Job.20Failure/near/223865005:

Dear @RalfJ and @oli\n\nIt would appear that the Miri cron job build failed. Would you mind investigating this issue?\n\nThanks in advance!\nSincerely,\nThe Miri Cronjobs Bot

This PR changes that to:

Dear @RalfJ and @oli
It would appear that the Miri cron job build failed. Would you mind investigating this issue?
Thanks in advance!
Sincerely,
The Miri Cronjobs Bot

Thanks for the PR! I don't like the way this is "de-indented" compared to the flags, but I don't see anything obviously better either, so let's go with this for now.
@bors r+

Thanks for the PR! I don't like the way this is "de-indented" compared to the flags, but I don't see anything obviously better either, so let's go with this for now.
@bors r+

:pushpin: Commit a5d7ae5816e22dcd37989e45ca3d36e6366e1c53 has been approved by RalfJung

:pushpin: Commit a5d7ae5816e22dcd37989e45ca3d36e6366e1c53 has been approved by RalfJung

:hourglass: Testing commit a5d7ae5816e22dcd37989e45ca3d36e6366e1c53 with merge 5483ed5836b7686db9115e66bb9374308bc0d873...

:hourglass: Testing commit a5d7ae5816e22dcd37989e45ca3d36e6366e1c53 with merge 5483ed5836b7686db9115e66bb9374308bc0d873...

:sunny: Test successful - checks-actions
Approved by: RalfJung
Pushing 5483ed5836b7686db9115e66bb9374308bc0d873 to master...

:sunny: Test successful - checks-actions
Approved by: RalfJung
Pushing 5483ed5836b7686db9115e66bb9374308bc0d873 to master...


add OpenCV3

@vrabaud it would be best if you could pull from an upstream tag or something rather than this:
https://github.com/ros-gbp/opencv3-release/blob/master/tracks.yaml#L68
It's not particularly traceable when the trail of where the code came from ends with a directory on your computer. If you give me the appropriate upstream version I can figure out how to do a normal release with that. (I know it's slower that way, but it is better for transparency).
Thanks.

Right, that's why it's explained on the wiki: http://wiki.ros.org/opencv3#Building_the_Package
You can also fix https://github.com/ros-infrastructure/bloom/issues/259 . Right now, I would have to pull the full 300M OpenCV repo and that's very slow :(

Well if you don't mind, I'd like to rebloom it. After all very slow != impossible. I'll just go get a cookie while it's running :cookie:.
Unfortunately I don't have time to look into ros-infrastructure/bloom#259 at the moment since it isn't simply add --depth 1 to all git clone calls (I'd need to check on versions and what not and also test it).


how can I stop keyboard.Listener from grabbing all keyboard input even when it doesn't hold focus?

I'm running the listener like this:
with keyboard.Listener(on_press=on_press, on_release=on_release, suppress=True) as listener:
but it grabs all keyboard input, blocking out other apps, until the with block ends. 'I'm doing a push to talk' type of app, and I need suppress True to stop echoing spacebar autorepeating.
Is there a way to allow other apps to use the keyboard when  they are given focus?
Im running on Ubuntu with sounddevice version 0.4.6

keyboard.Listener is not part of the sounddevice module.
You should probably ask at the issue tracker of the keyboard module?


weird gpu memory usage

Description
The script below will eat all gpu memory gradually in the first epoch. Sometimes it will cause OOM error and crash.
Sometimes it just keep running, but at the 2rd epoch, the gpu memory and cuda core usage will drop to a very low level suddenly.
Environment info (Required)
ubuntu 18.04 cuda91
nvidia driver 390.48
nvidia 1080ti gpu card
mxnet-cu91 1.2.0 from pypi (with or without mkl)
Minimum reproducible example
#!/usr/bin/env python3
import time
from multiprocessing import cpu_count

import mxnet as mx
from mxnet import autograd, gluon
from mxnet.gluon.data import dataset
from mxnet.gluon.data.vision import transforms
from mxnet.gluon.model_zoo import vision

CPU_COUNT = cpu_count()
mx.random.seed(42)  # set seed for repeatability

class ImgDataSet(dataset.Dataset):
    def __getitem__(self, idx):
        return mx.nd.random.uniform(shape=(320, 320, 3)), 0

    def __len__(self):
        return 50000

transformer = transforms.Compose([
    transforms.Resize(224),
    transforms.ToTensor()])

train_dataset = ImgDataSet()
train_dataset = train_dataset.transform_first(transformer)

batch_size = 16
train_data_loader = mx.gluon.data.DataLoader(
    train_dataset, batch_size, shuffle=True, num_workers=CPU_COUNT)

# construct and initialize network.
ctx = mx.gpu() if mx.test_utils.list_gpus() else mx.cpu()

net = vision.get_mobilenet_v2(1.0, ctx=ctx, classes=1000)
net.hybridize()
net.initialize(mx.init.Xavier(factor_type='out'), ctx=ctx)
# define loss and trainer.
criterion = gluon.loss.SoftmaxCrossEntropyLoss()
trainer = gluon.Trainer(net.collect_params(), 'Adam', {
    'learning_rate': 0.1, 'wd': 4e-5})

epochs = 50
for epoch in range(epochs):
    total_train_loss = mx.nd.zeros(1, ctx=ctx)
    start = time.time()
    training_samples = 0
    for batch_idx, (data, label) in enumerate(train_data_loader):
        data = data.as_in_context(ctx)
        label = label.as_in_context(ctx)

        with autograd.record():
            output = net(data)
            loss = criterion(output, label)
        loss.backward()
        trainer.step(data.shape[0])
        total_train_loss += loss.sum()
        training_samples += data.shape[0]
        if batch_idx > 0 and batch_idx % 500 == 0:
            elapse = time.time() - start
            # uncomment this line will prevent this error to happen.
            # print(total_train_loss.asscalar()/training_samples)
            print('{}:[{}] samples/s in train'.format(batch_idx,
                                                      training_samples/elapse))

If uncomment the asscalar line, the code run well but become slower and slower.
0.014223459476006483
500:[310.3343436613137] samples/s in train
0.0071204068301083685
1000:[138.02850671172925] samples/s in train
0.004750276866712068
1500:[117.94254639226769] samples/s in train
0.0035655089821117272
2000:[109.79963955911568] samples/s in train
0.002856015301093918
2500:[105.23927686823332] samples/s in train
0.0023851065745317153
3000:[102.53342911171562] samples/s in train
0.0020499249861193397
3500:[100.58626881090342] samples/s in train
0.0017987247467994452
4000:[99.1741771153156] samples/s in train
0.0016033576875494682
4500:[98.11294445964649] samples/s in train
0.0014470566084231885
5000:[97.28087273484665] samples/s in train
0.0013191592929537048
5500:[96.63322991265628] samples/s in train
0.0012125678905505494
6000:[95.973076875728] samples/s in train
3.9965115086047233e-05
500:[138.14346868877303] samples/s in train
3.9967515698441495e-05
1000:[110.0120848330557] samples/s in train
3.996974652723659e-05
1500:[102.58358700612278] samples/s in train

Hello @dwSun, thanks for submitting your issue. There is an open discussion forum about mxnet on discuss.mxnet.io - you could post your query there to learn more.

@dwSun MXNet is built using asynchronous operations. When you load data, run a forward pass or backward pass, the operations are enqueued on the MXNet backend and executed when the parent dependencies are available.
With your current script, there is no blocking operations, so the training runs through your epoch and keep adding "copy to GPU" operations. These operations don't have parent dependencies and can be executed immediately.  the actual training isn't completed when you reach the end of your epoch loop. After a few epoch you will clog up your GPU memory.
print(total_train_loss.asscalar()/training_samples)
.asscalar() is .asnumpy()[0], this causes a synchronous operation to copy the memory back to the CPU. When you add this line, the training isn't "slow" it is the normal speed, because every 500 iterations, your network is going to wait for the computation to be completed and return the result to the CPU. If your dataset is small and fit in GPU memory, you can have a mx.nd.waitall at the end of your epoch, but that means your entire dataset will be copied to GPU. This makes it pretty fast since at the beginning of each batch the data is already available. However you might run into OOM error, in that case you can for example keep track of your loss using loss_acc += loss.sum().asscalar() and forcing a copy to CPU on every batch.

@ThomasDelteil, thanks for explain that.


Get nextContinuationData of YT_MUSIC

Question
Hi, thanks for the awesome package!
iam realy confuse regarding pagination workflow
i just want to get nextContinuationData and send to the next request
has_continuation and getContinuation is working great, but i want make a pagination per request
may someone can tell me how to do that?
Thanks a lot
Other details
No response
Checklist

[X] I am running the latest version.
[X] I checked the documentation and found no answer.
[X] I have searched the existing issues and made sure this is not a duplicate.
[X] I have provided sufficient information.

Heya!
I assume you are handling pagination yourself, right? If so, then all objects such as nextContinuationData, reloadContinuationData and so forth are usually available as either a ContinuationItem YTNode or a property of a node (as seen in this node).
If you're dealing with ContinuationItem YTNodes, which are always at the very end of a list then simply calling their navigation endpoint would do:
const next_page = await target_continuation.endpoint.call(yt.actions, { parse: true, client: 'YTMUSIC' });
console.log(next_page);

If it is some sort of list that requires clicking a pseudo-button/text to load more items then it probably will be available as a token rather than a full ContinuationItem node. In this case, you can use Actions#execute to call it:
/**
 * NOTE:
 * I'm using the "/next" endpoint here but it could also be "/browse".
 * Since I have no context on what you're trying to accomplish we'll use "/next".
 */
const next_page = await yt.actions.execute('/next', {
  continuation: TOKEN_GOES_HERE,
  client: 'YTMUSIC',
  parse: true
});

console.info(next_page);

I don't really know what kind of list you're trying to retrieve continuations from, so pardon me if this was not helpful. More context would be appreciated though.
Have a great day/night!

Hi @LuanRT
Thank you for the quick reply
Iam so sorry, i still didnt get it
let say i search for songs like this:
        const results = await yt.music.search(req.body.q, {
          type: req.body.shelf
        });

Then how i can get continuations token coz i want to send in the next request with this token then load item from the next page.
Thanks you before

Ah! Thanks for clarifying.
That should be simple, here's a small example which you can use as reference:
import { Innertube, YTNodes, Parser } from 'youtubei.js';

(async () => {
  const yt = await Innertube.create();

  const search = await yt.music.search('Mac Miller', { type: 'album' });

  const shelf = search.page.contents_memo.getType(YTNodes.MusicShelf)?.[0];

  let continuation_token = shelf.continuation;

  if (!continuation_token)
    throw new Error('Continuation not available');

  // Get the next page using the token:

  const page = await yt.actions.execute('/search', {
    continuation: continuation_token,
    client: 'YTMUSIC'
  });

  const data = page.data.continuationContents.musicShelfContinuation;

  const results = Parser.parse(data.contents).array().as(YTNodes.MusicResponsiveListItem);
  continuation_token = data?.continuations?.[0]?.nextContinuationData?.continuation;

  console.info(results, continuation_token);
})();

Amazing, thank you so much @LuanRT , will try soon


improve words choice for clarity, fix missing '.' in sentences.

Minor changes for the usage of "isn't" or "aren't" to explicitly say "is not" or "are not."
Minor fix of the missing "." at the end of the sentences.
Small suggestions on wording choices.

@microsoft-github-policy-service agree

@WilliamDAssafMSFT
Can you review the proposed changes?
Important: When the changes are ready for publication, adding a #sign-off comment is the best way to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

Thanks @imalfro for taking the time to provide these edits, and @Court72 for the revisions. Contractions are ok generally, some of these make sense. We appreciate the careful attention here.
These commits will be taken into an internal PR and published as part of doc update already in flight. You should see the changes live in the next day or so.

By opening a pull request, you're helping the authors add your approved change directly to the documentation. When the article is updated with your commit, GitHub will list you as an article contributor.
Thanks again for this feedback, @imalfro. Your suggestions help improve our documentation.


fix(cloudcommon): remove unnecessary  debug info

What this PR does / why we need it:

[x] Smoke testing completed

Does this PR need to be backport to the previous release branch?:
NONE

/area cloudcommon
/cc @ioito @zexi

/lgtm


24_04_28_18_01_00 lots of work to sparse_dict

Changes made up until 24_04_28_18_07_00 on ubuntu. Mostly on sparse_dict, including divvying-up of the old sparse_dict module into separate modules, and building/refurbishment of the corresponding test.

OK!

merging with main 24_04_28_18_33_00


design Q: promises, angular-meteor 'reactivity', & Meteor.methods()

I got used to building my ionic+angularapps with promises, but now that I've switched to Meteor + angular-meteor#1.3.7 I'm discovering it's a lot of work to convert reactivity to promises.
With promises, I got used to knowing exactly when a change has completed -- and I can do the next thing. But with angular-meteor 'reactivity', the timing of backend changes don't seem to be so 'exact'. I can't help but think that I'm doing this wrong. Is it possible that what I used to do with promises should now be done inside autorun()? Am I supposed to let the $digest loop do more work?

# ./posts/post.controller.coffee
PostCtrl = ($scope, $q)->
  ReactiveContext = $reactive(this).attach($scope)
  vm = this

  # from ng-click=".onComment($event, $post, $comment)"
  $scope.onComment = ($ev, post, comment)->
    return vm.postComment(post, {message:comment})
    .then (comment)->
        console.log(["success",comment])
    , (err)->
         console.log(["error",err])

  # call Meteor.method with promises
  vm.postComment = (post, options)->
      comment = options.message
      from = Meteor.user()

      dfd = $q.defer()
      ReactiveContext.call 'Post.postComment', post, comment, from, (err, result)->
        if err
          return dfd.reject(err)
        dfd.resolve( Comments.findOne(result) )
        console.log ['Meteor::Post.postComment OK']
      return dfd.promise
  return vm

PostCtrl.$inject = ['$scope', '$q']

# ./server/comments.model.coffee
Comments = new Mongo.Collection('comments')

# Post hasMany Comments
Meteor.methods({
  'Post.postComment': (post, comment, from)->
    comment = {
      type: "Comment"
      ownerId: from._id
      postId: post._id
      createdAt: new Date()
      comment: comment
    }
    Comments.insert( comment )
})

Is this a "better" pattern for angular-meteor?
# ./posts/post.controller.coffee

  [...]
  vm.postId = $stateParams.id
  ReactiveContext.helpers {
    'comments': Comments.find({ postId: ReactiveContext.getReactively('vm.postId') })
  }
  ReactiveContext.autorun (tracker)->
    # using deep watch here
    comments = ReactiveContext.getReactively('comments', true)
   # do "promise-y" things here

@mixersoft
Instead of this:
ReactiveContext.helpers {
    'comments': Comments.find({ postId: ReactiveContext.getReactively('vm.postId') })
  }
  ReactiveContext.autorun (tracker)->
    # using deep watch here
    comments = ReactiveContext.getReactively('comments', true)
   # do "promise-y" things here

You can use this:
// js because I'm not familiar with coffeescript

vm.postId = $stateParams.id;

vm.helpers({
  comments: () => Comments.find({ postId: vm.postId })
});

And now under vm.comments you have fetched data.

ReactiveContext is just a reference to vm so it's basically the same and you don't have to use it like this:
ReactiveContext = $reactive(this).attach($scope);
// just
$reactive(this).attach($scope);

@mixersoft please reopen if that haven't solved your problem

@kamilkisiela  I understand your comment, but it doesn't really address my question - which is about using promises to tell angular when something was changed by the angular-meteor binding instead of just discovering the change while in the $digest loop.
In the example above, I know how to get Comments for a Post.Id. But, I want to know when a Comment has been added from somewhere else, and then grab the commenter's username and face pic. or some other attribute from a FK lookup.
I think my use case is related to data de-nomalization in Meteor. For example, I want to save a Meteor userId as a FK in a Meteor doc, and grab the username and face pic from Meteor.users  when I display the doc. I can do it once when the doc is retrieved (and the code is a little cleaner), or I can do it every time in the $digest cycle. Is this an un-Meteor way to do things?
Q: Should I just assume that it costs me nothing to do a FK lookup to get a data attribute from mini-mongo?
However, I did discover that you can use plain Meteor.call() with angular $q promises to get changes from Meteor into the angular world, i.e. 3 way-binding.

@mixersoft  probably best to move the issue to the meteor forum with an angular tag. It will be great to have that discussion there regarding recommended patterns with angular and so. Ping here when you opened the topic.


The slider in the collection browser responds to any keystroke :)

What happened?
I found a bug while browsing my collection. When you open a particular NFT in the beta version, there is a slider that simply takes you to the next NFT in that particular collection - that's great, I like it a lot. I like it up until I press any key on my keyboard it automatically skips the swipe to the next NFT. I figured it out by accident, I wanted to copy the text to my NFT (ctrl+c)
Please reproduce in steps
I figured it out by accident, I wanted to copy the text to my NFT (ctrl+c) and it automatically skips the swipe to the next NFT
Expected Behavior
I've expected ... that the scroll bar will only respond to a mouse command or pressing an arrow on the keyboard
What browsers are you seeing the problem on?
Firefox
Are you logged in?
Yes
At which address did you encounter bug?
beta.kodadot.xyz
Screenshots
video here
https://user-images.githubusercontent.com/81104034/148569390-f2d1b850-b12d-4d9a-a124-8ae11e5217bc.mp4
Relevant log output
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

May @kkukelka

May @kkukelka can fix

I'll take a look at it! 👍

Perfect!!! <3


Add configuration flag to hide kong headers

Hi!
If I currently do a kong request, the response contains the following headers:
'server': 'Apache-Coyote/1.1',
'via': 'kong/0.6.1',
'x-kong-proxy-latency': '0',
'x-kong-upstream-latency': '79'}
Server is the value from the upstream server, the rest is set by kong. While this is nice for debugging, I would prefer to hide those values from the public.
I would be nice to have a configuration flag to turn the exposure off.
Cheers
Ruwen

+1

This also overlaps with #324, just in case someone is looking into it.

I suspect others have already found work arounds for this but the simplest I have found is to add the following statements to the /etc/kong/kong.yml file to modify the nginx configuration.
        # Add additional response headers
        header_filter_by_lua_block {
           kong.header_filter()
           ngx.header["Server"] = nil
           ngx.header["Via"] = nil
           ngx.header["X-Kong-Proxy-Latency"] = nil
           ngx.header["X-Kong-Upstream-Latency"] = nil
        }

Where would this go in the kong.conf for 0.9.1?

Try modifying /usr/local/share/lua/5.1/kong/templates/kong/templates/nginx_kong.lua and restart Kong.

Hmm we run Kong on EC2 instances and simply install the kong rpm. Of course we can write some script, package it and let it do this modification. But it would be nice if there is a cleaner way.

The response transfer plugin can only remove the X-Powered-By header but all others seem to be added later.
@elruwen You can have a look at https://github.com/Mashape/kong/issues/1842#issuecomment-262719479 perhaps this is a better way to solve it.

#2259 was just merged and allows to do this thanks to @bungle. Expect this to be released in the coming weeks. Thanks!

I tried adding above mentioned configuration nginx_kong.conf file and restarted the kong server in the meanwhile the file got overwritten with the default configuration and my changes disappeared.
Please help me starting kong server with custom nginx_kong.conf.

We are doing currently the following:
There is the file /usr/local/share/lua/5.1/kong/templates/nginx.lua (the exact path might be different on your system).
We added to the http section the following lines:
    more_clear_headers 'Server';
    server_tokens off;

I found solution for Kong 1.4.X (using database deployment, meaning there's no kong.yaml)
Execute elruwen's solution by editing the following file:
sudo vim /usr/local/share/lua/5.1/kong/templates/nginx.lua

You will find something like this:
http {
>  if #proxy_listeners > 0 or #admin_listeners > 0 then
    include 'nginx-kong.conf';
> end
}

Modify it, by adding elruwnen's line, like this (if you add the line server_tokens off; kong won't start):
http {
>  if #proxy_listeners > 0 or #admin_listeners > 0 then
    include 'nginx-kong.conf';
> end
more_clear_headers 'Server';
}

Go and edit your kong.conf file:
sudo vim /etc/kong/kong.conf

Search for the headers section and edit it:
From this:
headers = server_tokens, latency_tokens

to this:
headers = off

restart kong

How to remove the headers server:0.13.1 version

How to remove the headers Server: kong/2.0.0 and use our own certificat in nginx

@Tryptich @Kishorelk The answer from @Edenshaw is correct. See the documentation for the headers entry at: https://docs.konghq.com/2.1.x/configuration/#headers


On the numpy version which may cause exception

Hi @yl-1993, I've reproduced an exception when using numpy version 1.20+/1.19+ for test_cluster_det_ms1m.sh. The exception would be incompatible size when dumping KNN result to npz file:
setting an array element with a sequence. the requested array has an inhomogeneous shape after 2 dimensions. the detected shape was (584013, 2) + inhomogeneous part.

By changing numpy version to 1.18 could solve this issue. Not sure what is the root cause but could take time to modify requirements.txt.

Also, it may be noticed that the dropbox storing the re-aligned face images https://www.dropbox.com/s/wpx6tqjf0y5mf6r/faces_ms1m-refine-v2_112x112.zip?dl=0 cannot be accessed. Could you please help to take a look if available? Thanks.

Also, it may be noticed that the dropbox storing the re-aligned face images https://www.dropbox.com/s/wpx6tqjf0y5mf6r/faces_ms1m-refine-v2_112x112.zip?dl=0 cannot be accessed. Could you please help to take a look if available? Thanks.

hi, Have you run the part1_test.bin test suite yet? Can I ask you some questions? I use gcnds model

@Vivecccccc  6 months too late but same problem, this repository is not too organized.


Hyperdrive mounts between browser tabs not working

I'm testing and trying to mount a hyperdrive created in another tab but the mount shows up empty.
In both tabs I create a new hyperdrive, wait both to be ready, write some files to drive2, get drive2.key.toString('hex') from tab 2 and try to mount in tab 1 with drive1.mount('ext', Buffer.from(drive2_key, 'hex')). But I get nothing in the mount. I can see from dev tools that I have an open wss connection to hyperswarm.mauve.moe and messages are being passed.
If I try this within one tab, everything works perfectly, so the problem seems to be related to hyperswarm?
If I understood README correctly, hyper-sdk is supposed to do everything for me behind the scenes, so I don't need to replicate the drives myself?
How can I debug this further?

Is one of these tabs using private browsing? Sometimes that can affect things.
Also are both of these tabs on the same domain and using the same storage?
Generally speaking only one instance of the SDK should be running per domain at a time since it can lead to errors if two tabs try to modify the same storage.
Setting some breakpoints in the peer-add events might be useful to see if you're getting WebRTC connections in.

Hyperdrive mounts don't exist in the latest version


error on a fixture : Prolights cromospot 300

hi, yesterday i have use that light and i had have found an error on Person1 Mode : two channels are inverted
Now is ok

Coverage: 28.078%. Remained the same when pulling 6d7e183caf95fc337ac5f51d864805f3ed5fd215 on Tolmino:master into 5ec2f1deb21c6aba3fc2b262b7c5b9555bc8c6fc on mcallegari:master.

Thanks 👍


[FP]: Undertow-core CVE-2022-2053 reported after fix in 2.2.19.Final

Package URl
<EMAIL_ADDRESS>CPE
cpe:2.3:a:io.undertow:undertow-core:2.2.19.Final:*:*:*:*:*:*:*
CVE
CVE-2022-2053
ODC Integration
{"label"=>"Maven Plugin"}
ODC Version
7.1.1
Description
From the report: "This flaw was fixed in Undertow 2.2.19.Final, Undertow 2.3.0.Alpha2."

This has been resolved by an update to the upstream vulnerability source


Daily Solar Radiation Sensor

Is your feature request related to a problem? Please describe.
Both this and the core weatherflow integrations include solar radiation in W/M2
I need this data to aggregate over the day like the sensor you have created for Precipitation Today
Describe the solution you'd like
I'd like a additional sensor that holds the running total for total solar radiation so far today. It should reset daily.
Describe alternatives you've considered
I have setup a Utility Meter Helper that calculates this but I keep getting a very strange issue where the solar radiation source data does not seem correct.
Additional context
See below for a history chart that shows this weirdness. Data highlighted is from today. The data is too perfect. Something is wierd.

If I overlay data from Utility Meter for daily total solar radiation - the issue is highlighted

The more I think about this - maybe its a bug not a feature request...

This all comes down to if this is setup as MEASUREMENT sensor or a TOTAL_INCREASING State Class. A sensor cannot be both. For the rain sensor it is obvious that this is a Total Increasing per day, but not for this sensor as it fluctuates over the day how much your Solar rad is.
I have Solar Power on my house and I like to compare the Solar Radiation to actual Watt's produced in realtime. If I had this as Total Increasing I could not do that.
The only solution is either create a new Sensor in the Integration with the same input, but State Class set to TOTAL_INCREASING or do as you do, and create a Utility Helper. (I still need to figure out how exactly to configure this, to meet your use case)

Mine also has some strange curves some days. We have had a very soon May until now (Not normal for Denmark 😎 ) As you can see from the below 4 1/2 history, the curves are not the same. Some are more rounded than others. Clouds ofcourse affect this curve very much.

Then the issue might be upstream with the tempest itself.
Any chance you could create a utility meter helper off your solar radiation to see if you get the same issue as me on days with perfect data?

I have setup a few with different configs now. They need a day or two to populate data

Thank you. Really appreciate your help. I have a feeling the data comes through strange when the sky is clear/sunny.
I will also monitor myself. Speak soon.


FIPS 140-3 - Required KAT for SHA2-256 appears to be missing

3.0, 3.1, 3.2 FIPS Provider -> self_test_data.inc has the below:
static const ST_KAT_DIGEST st_kat_digest_tests[] =
{
{
OSSL_SELF_TEST_DESC_MD_SHA1,
"SHA1",
ITM_STR(sha1_pt),
ITM(sha1_digest),
},
{
OSSL_SELF_TEST_DESC_MD_SHA2,
"SHA512",
ITM_STR(sha512_pt),
ITM(sha512_digest),
},
{
OSSL_SELF_TEST_DESC_MD_SHA3,
"SHA3-256",
ITM(sha3_256_pt),
ITM(sha3_256_digest),
},
};
As per 10.3.A Cryptographic Algorithm Self-Test Requirements of FIPS 140-3 Implementation Guidance,
"if the module implements a SHS function (FIPS 180-4), the following shall be the minimal requirements for SHS algorithms:
•	a CAST for SHA-1 is required;
•	a CAST for SHA-256 is required;
•	a CAST for SHA-224 is required if SHA-224 is implemented without SHA-256;
•	a CAST for SHA-512 is required;
•	a CAST for SHA-512/224 or SHA-512/256 is required if the SHA-512 CAST is not performed;
•	a CAST for SHA-384 is required if SHA-384 is implemented without SHA-512.
It would appear that the SHA-256 KAT is missing.  There is a SHA3-256 KAT, however FIPS 140-3 requires that there be a SHA2-256 KAT to be compliant.

We have a HMAC-SHA-256 self test so we should still be compliant under the 10.3.B  section of the FIPS 140-3 Implementation Guidance.

@slontis might want to further comment on this.

Please note that the HMAC-SHA-256 KAT is happening in integrity_self_test() function.

Yes, @t8m is correct - you only have to do it once.
@t8m there may be a requirement to test the algorithm used in the integrity check before using it. Can you read the 140-3 IG and see it this is correct..

I think that I came across this requirement

@t8m there may be a requirement to test the algorithm used in the integrity check before using it. Can you read the 140-3 IG and see it this is correct..

Yes, AFAIK such requirement is there and we comply with it.

The relevant statement in the FIPS 140-3 IG is the following on page 70.
"All underlying approved algorithms used within a higher-level algorithm shall be self tests, either with their own self test or as part of a self test for the higher level algorithm"

@corkyduke - I think this can be closed unless you have other questions?

Thanks for all of the analysis and assistance. I will close it.


react-real-time - Replaced the '@microsoft/sp-client-preview ' package with '@microsoft/sp-webpart-base'

Q
A

Bug fix?
yes

What's in this Pull Request?
Replaced the '@microsoft/sp-client-preview ' package with '@microsoft/sp-webpart-base' for the BaseClientSideWebPart definition according to the SPFx Drop 5 release notes (https://github.com/sharepoint/sp-dev-docs/wiki/Release-Notes-Drop-5).
Resolve this error mentioned here http://sharepoint.stackexchange.com/questions/199426/the-value-for-webpart-context-configurestart-must-not-be-undefined

thx Francky.


shouldActivateBackToTop also checks when JS loads

Additional check if backToTop button should be active when JS loads.

Hello!
Will it have any special meaning? After all, the script is already in: $(document).ready(function() {

Got it, it really will be useful, thanks!


the PS to apply label to SPO site used wrong cmdlet

Hi for code sample below to apply label to all SPO sites, we should use Set-SPOSite instead of Set-SPOTenant.
$sites | ForEach-Object {Set-SPOTenant $_.url -SensitivityLabel $Id}

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 0d40ac52-6f02-b0f3-0c45-9f359473703b
Version Independent ID: 108355a9-05a3-4d17-c417-c4d5664a4506
Content: Use sensitivity labels with Microsoft Teams, Microsoft 365 groups, and SharePoint sites - Microsoft 365 Compliance
Content Source: microsoft-365/compliance/sensitivity-labels-teams-groups-sites.md
Service: o365-seccomp
GitHub Login: @cabailey
Microsoft Alias: cabailey

@frankchen76  Why did you conclude that we should use Set-SPOSite instead of Set-SPOTenant - did you try as documented and it didn't work?  See similarly posted issue https://github.com/MicrosoftDocs/microsoft-365-docs/issues/4611 and the response from the PM owner.

that's interesting. When I looked at the PS, the script was trying to enumerate all site collections. $sites | ForEach-Object {Set-SPOTenant $_.url -SensitivityLabel $Id}. It looks like we need to use Set-SPOSite $.url -SensitivityLabel $Id to apply the label for each site instead of Set-SPOTenant. When I checked Set-SPOTenant which doesn't have -SensitivityLabel parameter. -SensitivityLabel is only available for Set-SPOSite. you know what, I actually run Set-SPOTenant and Set-SPOSite with -SensitivityLabel, both cmdlets actually worked. :-) which is very interesting. If Set-SPOSite and Set-SPOTenant both can do the same thing, I believe Set-SPOSite would make more sense. If we don't add additional description there, people will ask the same questions.

@frankchen76  We document "by design" and after it's tested/confirmed by engineering. What we document has support implications. If you can't get the documented command with Set-SPOTenant to work, you're guaranteed support from Microsoft - they will investigate and help you get it working, or report a bug to engineering to fix. If you use Set-SPOSite instead for this single command to apply a sensitivity label to multiple sites and it doesn't work, the first thing a Microsoft support engineer will ask you to do is try the command as documented. But if it works for you, feel free to use it. I have passed on your feedback to engineering but won't change the documentation until I have confirmation from them that Set-SPOSite as an alternative for this specific scenario also works and will be supported.

@cabailey, thank you so much for your note. I'll close this issue and info my customer to use Set-SPOTenant to follow the document.


add the "bucketSpareCapacity" function

This simple PR adds a new function called bucketSpareCapacity to be used to query an OutboundQueue regarding the "free slots" it has to accept more peers. It can be used by application code to implement services to check if we hit our subscription limit for a particular <queue, bucket> combination.
cc @edsko @dcoutts @avieth

Using 'Nothing' to mean 'As much as you like' is a bit confusing semantically though

I do agree! Any suggestion? I guess we could create a separate type like:
data BucketCapacity = CapacityUnlimited
                                    |  BucketCapacity !Int

It would act essentially as a non-polymorphic version of Maybe a, but with a more precise semantic. I'm easy!

It doesn't really return the capacity though, does it? It returns spare capacity?

It returns spare capacity?

Correct. So I guess we either name the type appropriately (I suck at names!) or we simply stick with the status quo (Maybe) :)

How about
data SpareCapacity = SpareCapacity Int | UnlimitedCapacity

:)

Do like! :)

@edsko ok, it should be semantically less confusing now! 😛


Add draw_chimera_bqm to docs

Add function added by #156 to the docs

@arcondello, noticed this function and example as I went through the file, tried it but it fails, and that brought me to this issue.
draw_chimera_bqm(BinaryQuadraticModel.from_qubo(Q), width=1, height=1)
/home/jpasvolsky/Work/MyVirtualEnvironments/dimod/lib/python3.5/site-packages/networkx/drawing/nx_pylab.py:565: MatplotlibDeprecationWarning: isinstance(..., numbers.Number)
  if cb.is_numlike(alpha):
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-4-4cb31be40579> in <module>
----> 1 draw_chimera_bqm(BinaryQuadraticModel.from_qubo(Q), width=1, height=1)

~/Work/MyVirtualEnvironments/dimod/lib/python3.5/site-packages/dwave/embedding/drawing.py in draw_chimera_bqm(bqm, width, height)
    101     # Draw your graph over the complete graph to show the connectivity.
    102     draw_chimera(G, node_size=node_size, linear_biases=bqm.linear, quadratic_biases=bqm.quadratic,
--> 103                      width=3)
    104     return

~/Work/MyVirtualEnvironments/dimod/lib/python3.5/site-packages/dwave_networkx/drawing/chimera_layout.py in draw_chimera(G, **kwargs)
    241
    242     """
--> 243     draw_qubit_graph(G, chimera_layout(G), **kwargs)
    244
    245

~/Work/MyVirtualEnvironments/dimod/lib/python3.5/site-packages/dwave_networkx/drawing/qubit_layout.py in draw_qubit_graph(G, layout, linear_biases, quadratic_biases, nodelist, edgelist, cmap, edge_cmap, vmin, vmax, edge_vmin, edge_vmax, midpoint, **kwargs)
    167             if midpoint is None:
    168                 midpoint = (global_vmax + global_vmin) / 2.0
--> 169             norm_map = mpl.colors.DivergingNorm(midpoint, vmin=global_vmin, vmax=global_vmax)
    170
    171             node_color = [cmap(norm_map(node)) for node in node_color]

AttributeError: module 'matplotlib.colors' has no attribute 'DivergingNorm'


Security Test

This PR has been reported for potential abuse to GitHub

Sorry for this, this PR is related to Bugcrowd Submission


TypeError: Audio.init() got an unexpected keyword argument 'source'

Loading text model from ./models\text_2.pt to cuda
Loading coarse model from ./models\coarse_2.pt to cuda
Loading fine model from ./models\fine_2.pt to cuda
Launching Bark UI Enhanced v0.7.4 Server
Traceback (most recent call last):
File "C:\Bark\bark_win\bark-gui\webui.py", line 366, in
swap_audio_filename = gr.Audio(label="Input audio.wav to swap voice", source="upload", type="filepath")
File "C:\Bark\bark_win\installer_files\env\lib\site-packages\gradio\component_meta.py", line 159, in wrapper
return fn(self, **kwargs)
TypeError: Audio.init() got an unexpected keyword argument 'source'
The gradio is version 3.50.2
Someone can help me please ?

Uninstall gradio with and install version 3.50.2
pip uninstall gradio
pip install gradio==3.50.2

it got renamed to sources so rename source to sources


Cannot send invitation

Cannot send mail from http://blog.mydomain.com:2368/ghost/settings/users/ to user with newer TLD. (in this case, .rocks)
/home/ghost/ghost/core/server/mail.js:95
                if (data.error.errno === 'ENOTFOUND') {
                              ^
TypeError: Cannot read property 'errno' of null
    at EventEmitter.<anonymous> (/home/ghost/ghost/core/server/mail.js:95:31)
    at EventEmitter.g (events.js:180:16)
    at EventEmitter.emit (events.js:95:17)
    at DirectMailer.<anonymous> (/home/ghost/ghost/node_modules/nodemailer/node_modules/directmail/lib/mailer.js:138:30)
    at asyncCallback (dns.js:68:16)
    at Object.onanswer [as oncomplete] (dns.js:147:9)

cc @javorszky

Ohkay... right, sooo... that's the validator plugin throwing errors. I'll have a 4 day weekend, and will update the email module from 0.7.2 to whatever the latest version it is currently. Thanks for the bug report @lexpierce, and @novaugust for the cc. :)

Just a heads up, Mandrill is being difficult: https://github.com/andris9/Nodemailer/issues/412 and https://twitter.com/javorszky/status/584738416564244480
If anyone can tell me what I'm doing wrong, and point me in the right direction, it'd be appreciated. :)

Hey all,
So I've done a bit of research involving buying http://javorszky.rocks, setting that up as a domain alias in google apps, and configuring the email on that domain.
After that I invited myself on my new email address, and this is the result:

This is on latest master of Ghost, with zero modifications to anything. Latest commit at the moment is https://github.com/TryGhost/Ghost/commit/5525d0ea3420dd2b3b4f343ffad16913e8a0a566.
@lexpierce, I think it has more to do with DNS setup / propagation time than the email modules.

I'm going to close this, as if there's a mail sending problem it's not clear what it is at this point.  If anyone has anything else to add we can re-open.


use url encoded path in s3 request

use path url encoding on s3 path in http request body
the encoding done in the signature calculation already includes this step, so i think the signature is correct.

👍


Support mono on Docker-Alpine

Greetings,
I was wondering if you could help me get mono running on your OS. I really like the whole idea about a minimal, hardened OS -- and having a small footprint means a quicker feedback loop in dev.
I've been chatting with some Alpine OS committers on the IRC chat and they have built a package for mono http://pkgs.alpinelinux.org/packages?package=mono%25&repo=all&arch=x86_64 -- except it's still not quite working. Could you help me make it work, you think?
Right now it's crashing with: Error relocating /usr/lib/libmonosgen-2.0.so.1: __flt_rounds: symbol not found
Regards,
Henrik

Can you help provide some steps to reproduce or at the very least the docker run command and Dockerfile building the image? It looks like the mono package is only in edge testing - are you running the edge tag? It wont likely work on latest or 3.1.

Yes, I'm running edge. I'll supply a dockerfile in an hour or two.

Any thoughts on this?

I'd love to see this, I've got a small amount of .Net code that I use with edge.js to access with node, though it's very limited...
Not to mention ASP.NET/MVC vNext is much more open and a minimal image base is desired.

I have just created a alpine-mono base image with a usage example.
https://github.com/MattAitchison/alpine-mono

@MattAitchison awesome! What version do you have?

@luisrudge it should have version 4.0.1-r1

@MattAitchison awesome!

The recent 3.2 release of Alpine should now have mono in it. If there is still an issue with it, please reopen and let us know how to reproduce, we'll be happy to help.

FYI, Mono 4.0 says that they have only "rudimentary support for musl libc". I'm not sure what kind of bugs that may introduce, but just in case you can try my frolvlad/alpine-mono image, which is based on glibc hack and Arch Linux mono package.

Yes, I wouldn't put anything into production without a whole lot of testing first.
@andyshinn Thanks for the help in getting this far! I hope to use much more of Alpine with this in!

I'm not seeing the package when I try to install it with the latest 3.3 image:
/ # apk add --no-cache mono
fetch http://alpine.gliderlabs.com/alpine/v3.3/main/x86_64/APKINDEX.tar.gz
fetch http://alpine.gliderlabs.com/alpine/v3.3/community/x86_64/APKINDEX.tar.gz
ERROR: unsatisfiable constraints:
  mono (missing):
    required by: world[mono]

Am I missing something simple?

@MattKunze the mono package is in testing repository. Follow the official wiki about adding that repo to your /etc/apk/repositories. I didn't have a chance to try this package yet since the the hacky glibc version (referenced above) works fine for my project.

thanks! Added http://dl-4.alpinelinux.org/alpine/edge/testing to the list and it installed.
Now I guess to see if anything works... :)

Just tried to build something with coreclr
$ docker run -it kodj/alpine-aspnet bash
$ curl -sSL https://raw.githubusercontent.com/aspn…/…/dev/dnvminstall.sh | DNX_BRANCH=dev sh && source ~/.dnx/dnvm/dnvm.sh
$ dnvm

@trumhemcut did it work?

have a look at https://github.com/jumanjihouse/docker-dotnet for an experiment with docker + alpine + coreclr

@trumhemcut CoreCLR is off topic in this thread. Please stick to Mono on Alpine in this thread.

Is there any way to get mono on arm32v6? the current mono package is only for x86.

@AnderssonPeter you may find a lit of issues even if you get mono on arm in alpine, under docker.  There's a lot of room there for things not to work.  Starting with libraries than may have hard coded builds/bits for x86.


Order recent docs on 10.11 properly

It came to my attention recently that → into apps doesn't seem to leave files in the right order anymore under El Capitan. That's because they're no longer stored in the plists (.sfl files) in the order they were opened, but are stored in a random order.
Since they're stored as the new SFLListItem class (using a keyed archiver) in 10.11, which has a property order, we can use that to reorder the items.
I took inspiration from http://pastebin.com/dYxrGX7c

I hope I managed to rebase and force push quick enough before people could see my poor indentation? ... :worried:
I should really practice what I preach eh? ;-)

Also see https://michaellynn.github.io/2015/10/24/apples-bookmarkdata-exposed/

Looks good, works as advertised, and wipes out one of the edge cases I was worried about in the “sort by date” implementation.
My only concern is this:
Class SFLListItem is implemented in both /System/Library/Frameworks/CoreServices.framework/Versions/A/Frameworks/SharedFileList.framework/Versions/A/SharedFileList and /Applications/Quicksilver.app/Contents/Frameworks/QSFoundation.framework/Versions/A/QSFoundation. One of the two will be used. Which one is undefined.

Are you not content blindly sending messages to those objects without “knowing” what they are like I was? :stuck_out_tongue:

Are you not content blindly sending messages to those objects without “knowing” what they are like I was?

Hmmm… I didn’t see that error message!
Is there any way we can #import that header then, or is it totally private? Looks to me like it’s just a binary so I’d suggest totally private.
Perhaps we can make it a protocol, and say that the object is id <SharedFileListProtocol>, or maybe the best alternative is just to use a different name like QSSharedFileList
@tiennou what are your thoughts?

On 1 Feb 2016, at 02:56, Rob McBroom<EMAIL_ADDRESS><EMAIL_ADDRESS>wrote:
Are you not content blindly sending messages to those objects without “knowing” what they are like I was?

class-dump that framework, grab the header and drop that in External ? It might also be acceptable to just uncheck the implementation you provide, so the runtime warning goes away and the compiler finds its definition.
Changing the NSKeyedArchiver is prone to do weird things if ever we (or some Apple stuff) tries to unarchive some SFLListItem. I'm also interested if there's a behavioral difference between our code and the example code from the blog post (e.g. is it necessary to tell NSKeyedArchiver about that new class that it might already know about if we link against CoreServices ?).

class-dump that framework, grab the header and drop that in External ? It might also be acceptable to just uncheck the implementation you provide, so the runtime warning goes away and the compiler finds its definition.

Yeah I thought about that, but we actually do need to implement initWithCoder: in our code.
I tried just doing what Rob was previously doing and calling
float order = [item order]

but this was returning really weird results. Doing print (float) [item order] in the debugger gave the right number, but saving it to a variable then trying to use it was giving some obviously incorrect number. I figured it was maybe something to do with casting to the wrong type or 64/32 bit or something but I couldn't figure it out - so I left it as it is.
Just realised using a protocol won't work since we need to call
// Set up SFLListItem for unarchiving. See SFLListItem_Private.h
[NSKeyedUnarchiver setClass:[SFLListItem class] forClassName:@"SFLListItem"];

Maybe the thing to do is:

class dump and add the header file
subclass it as QSSFLListItem
override initWithCoder: and call [super initWithCoder:aCoder]; first?

is it necessary to tell NSKeyedArchiver about that new class that it might already know about if we link against CoreServices

They do it as well. See line 22-23
# Create the association for the class name
NSKeyedUnarchiver.setClass_forClassName_(SFLListItem, "SFLListItem")

OK @tiennou @skurfer
I've force pushed (sorry) but it's now much cleaner. A simple class dump and #import was all that was required :-)

I've force pushed (sorry) but it's now much cleaner.

No problem. I feel better about how it is now. After using the rough draft for a couple of days, I can’t believe I lived with the previous annoying behavior.


Support additional labels in the PodMonitor

This PR adds support for specifying additional labels in the PodMonitor configuration for Prometheus monitoring.
The new labels parameter allows users to customize labels applied to the PodMonitor resource.
This is useful for scenarios where podMonitorSelector is used to select specific PodMonitor resources based on their labels.

Please ensure your pull request adheres to the following guidelines:

[x] explain your motives to contribute this change: what problem you are trying to fix, what improvement you are trying to make
[x] Document your Changes in the CHANGELOG.md file of the respected chart as well as the Chart.yaml

Hello @anthonyroussel thanks a lot for participating in the community, could I ask you to rebase your changes and add the changelog entry to the chart.yml files ?

Hello @jCOTINEAU, thanks for the review.
I rebased my PR against master branch and added the changelog entry to the Chart.yml files.

Hello @anthonyroussel thanks again for your contribution! i added some tests and fixed a typo, those changes has been merged alongside yours in this PR
You are still author of the commits, but we usually do this because some of our tests does not run well with external contributors.
Huge thanks again!


tests: Add integ test for RDS database

Issue
Closes #225
Description of changes
Add integ test for RDS database
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

AWS CodeBuild CI Report

CodeBuild project: IDFModuleTestsInfrastructur-59JoFoVH64CI
Commit ID: c1a1b078c8164e6fa1f544ef1d6d76c32cb31f57
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: IDFModuleTestsInfrastructur-59JoFoVH64CI
Commit ID: 69b6e92098538e38ce5632669bc414d7512ac640
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: IDFModuleTestsInfrastructur-59JoFoVH64CI
Commit ID: d06ad0148ec92eb6f05e4ea8f71f5be5386d7dc1
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: IDFModuleTestsInfrastructur-59JoFoVH64CI
Commit ID: 69b6e92098538e38ce5632669bc414d7512ac640
Result: FAILED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

AWS CodeBuild CI Report

CodeBuild project: IDFModuleTestsInfrastructur-59JoFoVH64CI
Commit ID: 0789d200491c74d7f01e558f22e3c5243450f022
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository


"Copalnic-Mănăștur", "Berința", "Cărpiniș", "Copalnic", "Copalnic-Deal" and "Curtuiușu Mic" are listed in the drop-down menu

Description:
"Copalnic-Mănăștur", "Berința", "Cărpiniș", "Copalnic", "Copalnic-Deal" and "Curtuiușu Mic" are listed in the drop-down menu alongside its corresponding county and commune, according to the law
Precondition:
The website is up an running.

Step 1
Type "Copalnic-Mănăștur" in the search bar.

Expected results
The location is listed in the drop-down menu as "Sat Copalnic-Mănăștur, MARAMUREȘ (Copalnic-Mănăștur)" and "Comuna Copalnic-Mănăștur, MARAMUREȘ".

Step 2
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 3
Type "Berința" in the search bar.

Expected results
The location is listed in the drop-down menu as "Sat Berința, MARAMUREȘ (Copalnic-Mănăștur)".

Step 4
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 5
Type "Cărpiniș" in the search bar.

Expected results
The location is listed in the drop-down menu as "Sat Cărpiniș, MARAMUREȘ (Copalnic-Mănăștur)".

Step 6
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 7
Type "Copalnic" in the search bar.

Expected results
The location is listed in the drop-down menu as "Sat Copalnic, MARAMUREȘ (Copalnic-Mănăștur)".

Step 8
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 9
Type "Copalnic-Deal" in the search bar.

Expected results
The location is listed in the drop-down menu as "Sat Copalnic-Deal, MARAMUREȘ (Copalnic-Mănăștur)".

Step 10
Press "x" button.

Expected results
The location was deleted from the search bar.

Step 11
Type "Curtuiușu Mic" in the search bar.

Expected results
The location is listed in the drop-down menu as "Sat Curtuiușu Mic, MARAMUREȘ (Copalnic-Mănăștur)".


ceph-rgw-loadbalancer: Add haproxy stat socket permissions

Add ability to configure haproxy stat socket permission for use with haproxy-exporter.
Signed-off-by: Stanley Lam<EMAIL_ADDRESS>

jenkins test centos-container-update

jenkins test centos-non_container-switch_to_containers

bump


Feat request: Store progress in localStorage or indexedDB

First of all, thank you guys for making such a fun & easy to learn interactive game interface. I liked it a lot & I think the idea is brilliant.
I am just missing one feature on this site:
Currently after solving few levels if I close the tab & reopen it starts from the beginning & the progress is not shown.
Ideally, we should store the progress so that the next time someone opens it opens from the last level he/she was on & continue from there.
As there is already a reset button in the level dropdown so someone can reset their progress whenever they want if they want to start from the beginning next time.
One other option could be to show a popup on load when the user has some progress to ask them whether they want to continue from level 'n' or want to restart.

Ohh.. I realized that it works fine. I was doing it wrong. Closing this.


Docker container has an old youtube-dl

Please tick all applicable boxes.

[x] I am using Python 3.5 or higher (run python --version on the command line)
[ ] I have followed the official guides to install the bot for my system
[ ] I have updated my dependencies to the latest version using the appropriate update script

Which version are you using?

[x] The latest master version (1.9.8)
[ ] The latest review version

What type of issue are you creating?

[x] Bug
[ ] Feature request
[ ] Question

Description of issue
The latest build of the Docker container has an out of date youtube-dl, this means the bot can't interact with YouTube.
Steps to reproduce

Start the bot through the Docker container (I'm using kube kubectl run musicbot --image=justsomebots/musicbot:1.9.8)
Attempt to !play a song
Observe a youtube-dl regex error in the logs

Log file
ERROR: Signature extraction failed: Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/youtube.py", line 1210, in _decrypt_signature
    video_id, player_url, s
  File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/youtube.py", line 1121, in _extract_signature_function
    res = self._parse_sig_js(code)
  File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/youtube.py", line 1182, in _parse_sig_js
    jscode, 'Initial JS player signature function name', group='sig')
  File "/usr/lib/python3.6/site-packages/youtube_dl/extractor/common.py", line 972, in _search_regex
    raise RegexNotFoundError('Unable to extract %s' % _name)
youtube_dl.utils.RegexNotFoundError: Unable to extract Initial JS player signature function name
 (caused by RegexNotFoundError('Unable to extract Initial JS player signature function name',))

Possible solution
Rebuild the Docker container from further up the file, maybe by editing a comment before the youtube-dl package is installed. This will fetch the latest version.

Just run the update file lol

I don't appreciate the lol and that's not how Docker containers work. You must rebuild it from an earlier step because the deb (I presume) package needs updating. The old version is cashed in a layer. I could just build my own fixed container but I thought I'd try to help you out.

Our docker images are built automatically on each commit via Travis CI
The easiest solution for this problem would be, to set the entry point for docker to run the script that would run the update script and then run the actual run script. This would add some startup time. Anyway, a temporary fix, for now, would be to use the review image. review image should be working as it just been built a week ago. Youtube_dl was updated on Sep 17, 2018. master branch was built in August. The review branch currently does not have any significant difference from the master branch.

Excellent, thank you! That review tag does indeed work. I guess I was mistaken with the original issue, it just requires a new version, not rebuilding the container in some different way. I was under the impression that the RUN apk add... lines would be cached until a build step changed above those lines.

You can close this if you want or maybe leave it open as a warning about the current version until the new one is out.


The steam rate limit

Description
I am getting error
Reproduction Steps
Did all right
Errors
20:19:24 [Main] ERROR ThreadManager - The Steam Rate Limit has been reached. Please try again in a few minutes.
System Information
W10
Titan 1.6.1

You reached the Steam Rate Limit, can't be anything done against it except using a VPN or Proxy or waiting until you're no longer rate limited.


Bail on failed SSH command with information about what failed

@ehazlett @bfirsh @prologic WDYT?
Closes https://github.com/docker/machine/issues/1266
Signed-off-by: Nathan LeClaire<EMAIL_ADDRESS>

We can't just test for the exit code directly?

@prologic There's no cross platform way to do so that I am aware of (see http://stackoverflow.com/questions/10385551/get-exit-code-go, for instance).

Hmm I find this quite surpsingly really See Popen.returncode -- Unless the Python standard library has been lying to me this whole time!

Go really has no platform independent way of returning the exit status of a subprocess?

Go really has no platform independent way of returning the exit status of a subprocess?

AFAICT not really.  There are some "hack-arounds", and it's fairly easy to tell whether a process failed or succeeded, but I think the actual numeric codes are fairly platform-specific.

Weird but okay :)
James Mills / prologic
E<EMAIL_ADDRESS>W: prologic.shortcircuit.net.au
On Wed, Jul 15, 2015 at 11:29 AM, Nathan LeClaire<EMAIL_ADDRESS>wrote:

Go really has no platform independent way of returning the exit status of
a subprocess?
AFAICT not really. There are some "hack-arounds", and it's fairly easy to
tell whether a process failed or succeeded, but I think the actual numeric
codes are fairly platform-specific.
—
Reply to this email directly or view it on GitHub
https://github.com/docker/machine/pull/1501#issuecomment-121448737.

Nice!
Might be worth explaining a little clearer what's going on to make it less scary: "Machine ran a command on a host using SSH and it failed:"
Does the upstream error also explain how you might recover from this? (e.g. remove the host and try creating it again)

cc @ehazlett --- I'm not sure about this exact output format but I really want to get this in for the next release as our failing SSH commands right now and not spitting out much that's useful.

I think we can tweak the output later.  I agree it would be great to have this soon.
LGTM

Pending Travis, LGTM.  Should work with native and external SSH both.

LGTM


Add a solution to problem 632

The idea is to attach row information to each value and sort them by value. Then, we use two-pointer to determine the smallest range that all rows are presented. Using a priority queue to build the sorted values could lead to an O(N log k) time complexity; my implementation did not utilize the pre-sorted nature of the input lists.

Knowing that the lists are sorted, we can achieve better time and space complexity. We maintain one value from each list, ensuring the range includes a value from every list. We begin with the first value from each list. To move to the next range, we remove the smallest value and add the next element from its list. This continues until a list is exhausted and the smallest range encountered is the result. What does this process actually do? Notice that the values [left boundary of ranges] are popped according to the global sorted order. Indeed, the right boundary of the range at each step is the smallest possible value that still includes at least one element from each list, given the left boundary.


Added Windows specific config location

Windows by default does not set or use the $HOME environment variable. This commit adds a Windows specific branch which tries to locate the config file in %APPDATA%/erdtree/.erdtreerc instead
Fixes #152

For now I only did the minimal amount of changes required. Couple of remarks:

This adds a new dependency to the dirs crate, I am not sure if you are okay with that. But if we don't use a crate we have to resolve the %APPDATA% path manually, which might be error-prone
Maybe it also makes sense to exclude the $XDG_CONFIG_HOME path when compiling for Windows?

Not related to this feature, but in general when developing on Windows: I could not build the project intially, because of two compile errors:
let out = <str as AnsiEscaped>::truncate(&ln, window_width);
    |                               ^^^^^^^^^^^ use of undeclared type `AnsiEscaped`

After changing AnsiEscaped to just Escaped, everything worked and it compiled. I did not commit this change however, as I was unsure.
Also on Windows at least one unit test fails (not related to my changes, happens when just checking out the base repo). This is the output:
---- dirs_only stdout ----
thread 'dirs_only' panicked at 'assertion failed: `(left == right)`
  left: `"149  B ┌─ the_yellow_king\n447  B ├─ lipsum\n318  B ├─ dream_cycle\n1265 B data\n\n3 directories"`,
 right: `"143  B ┌─ the_yellow_king\n446  B ├─ lipsum\n308  B ├─ dream_cycle\n1241 B data\n\n3 directories"`', tests\dirs_only.rs:7:5

It seems on Windows some byte sizes are different? Not sure what the best way would be to fix this.

I am in agreement that the dirs crate is the way to go 👍
And yeah let's not compile $XDG_CONFIG_HOME for Windows.
As for the error that you fixed with AnsiEscaped to Escaped thank you. Someone yesterday ran clippy and submitted a PR with some errors that got merged because the entire CI test suite uses Ubuntu so the errors it caused on Windows went unnoticed. Again thank you for fixing that.
As for the failing test let's see what the workflow says once it's done running. I will be on a flight for a bit but am excited to get this merged in!


Training problems

Hi,
Sorry to bother you, I was wondering whether the program is able to train an entire folder of annotated images at once or would I have to train it image by image? Somehow when I do "./amf train folder_name" it says that my images in the folder are not annotated for some reason, but when I train it image by image by drag and drop, it does not throw that error. (All images along with the zip files are in the same folder that I gave it). I also tried giving it multiple images by simple drag and drop, but the program only seems to take the first image as input and ignores the rest.
I also get the following error for some images that I input, which I'm not exactly sure why, since I annotated them all in the same manner.
ValueError: class_weight must contain all classes in the data. The classes {2} exist in the data but not in class_weight.
Any insight would be much appreciated. Thank you so much for all your help!

Hello,
Yes, it does. The command is amf train <parameters> <images> which translates into amf train folder_name/*jpg in your case. If you only provide folder_name, it will assume folder_name is an image and throw an error.
The second error sounds more unusual to me. Once you confirm the first issue is solved, would you mind sharing an annotated picture that triggers the class_weight error? That would be great. Cheers.

Hi,
Thank you so much for your reply!
I have images of different species of plants colonized by AM saved in different folders, and while "*/jpg" did fix the problem for some of the folders, it did not for others. I was thus wondering whether there are any initial requirements to the images and whether that could have caused the problems? All of them are in jpg format but some are bigger than others. I also tried changing the image name, re-annotating, moving the images into the folder that worked, but neither of those helped, so I would really appreciate some insight! Thank you!
And I have attached an annotated figure that caused the program to throw the class_weight error. I think its possibly because I didn't assign any M- classes? But does that mean I have to use images with all classes present? I can try to do this for CNN1, but for CNN2 we don't necessarily see all four features (i.e. arbuscules, vesicles, intradical hyphae and the hypophodium) in one image, so I was wondering whether there's a trick around not having to have all classes annotated in one image?

Sorry for the long questions and thank you so much for all your help! I really appreciate it!

Hi,
Would you be able to share a picture and zip file for both situations (one
that is not used for training, and one that triggers class_weight error)? I
would need the actual images to test them on my machine (not a screenshot).
I suspect the second case to be a bug. I am not sure about the first case.
Please use either Dropbox or any other file sharing tool if the images are
too large for an email. That would be great!
Many thanks.
Best regards,
Edouard
Le mer. 9 févr. 2022 à 00:14, florak @.***> a écrit :

Hi,
Thank you so much for your reply!
I have images of different species of plants colonized by AM saved in
different folders, and while "*/jpg" did fix the problem for some of the
folders, it did not for others. I was thus wondering whether there are any
initial requirements to the images and whether that could have caused the
problems? All of them are in jpg format but some are bigger than others. I
also tried changing the image name, re-annotating, moving the images into
the folder that worked, but neither of those helped, so I would really
appreciate some insight! Thank you!
And I have attached an annotated figure that caused the program to throw
the class_weight error. I think its possibly because I didn't assign any M-
classes? But does that mean I have to use images with all classes present?
I can try to do this for CNN1, but for CNN2 we don't necessarily see all
four features (i.e. arbuscules, vesicles, intradical hyphae and the
hypophodium) in one image, so I was wondering whether there's a trick
around not having to have all classes annotated in one image?
[image: Screen Shot 2022-02-08 at 12 55 43 PM]
https://user-images.githubusercontent.com/98064133/153097214-cbb0fcd6-5a9b-41cc-bcde-b34570c0f518.png
Sorry for the long questions and thank you so much for all your help! I
really appreciate it!
—
Reply to this email directly, view it on GitHub
https://github.com/SchornacklabSLCU/amfinder/issues/16#issuecomment-1033191019,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AWXWWWCONQP5WLUMLOL5MJDU2GWXXANCNFSM5NZGUSDQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you commented.Message ID:
@.***>

Hi Edouard,
Thanks so much for your instant reply again! I really appreciate it!
Here's a google drive link to the images that work/don't work (for the annotations) and two images that throw the class_weight error:
https://drive.google.com/drive/folders/1GhKhNZZ-MTZcMIAudE1atHcl7v6VmRIR?usp=sharing
Thank you so much for your help again! Really appreciate it!
Sincerely,
Flora

Hi Flora,
Thanks for reporting this bug. I fixed the problem with class weights.
I never used AMFinder with fluorescent pictures before, hence I always had all classes in the CNN1 training dataset. The new amf will handle missing classes. Please download the new version, unzip it, then just replace the old amfinder_train.py with the new one (there is no need to reinstall amf). I could start training with images from both directories.
Please let me know if that works for you.
Cheers,
Edouard

Hi Edouard,
Thank you so much for all your help! I really appreciate it!
It works now!
Sincerely,
Flora

Hi Flora,
I am glad this was helpful. Good luck with your experiments!
Best regards,
Edouard


Must-Have: remove first 20 points from visualization of personality dimensions

Users cannot score lower than 20 points on any of the personality dimensions. If we remove the first 20 points from the personality visualization, then the differences in personality scores will become more visible. Currently, the shape of the personality graph tends to a rather balanced pentagon, even when personality scores are quite different from each other. This is probably a scaling issue that can be remedied by removing the first 20 points from the visualization.

I think if i am reading this right, it means that what we really mean is that the minimum score in a dimension is 10 and the maximum is 50 and we want to stretch that range so that it displays from 0 to 100.
I put in this formula
return Math.floor(((value - 10) / 40) * 100);

i think that does what you want, but let me know if not.

Looks good.

Can we close this if it looks good?

Verified Fixed


Adds server-side encryption to s3 puts

http://boto.cloudhackers.com/en/latest/ref/s3.html
Optionally adds encrypt_key to set_content_from_string(). This is required for writing to s3 buckets/keys that are protected with encryption at rest.

I'm unsure how to test this in your environment because you have to enable server-side encryption on the bucket/key. However, we have successfully tested it on our production systems. Thanks for your helpful package!

Coverage increased (+0.07%) to 98.529% when pulling 5feb6ba6f7dbef8aa9ecba4d65342cc41c5dfaae on lydiaauch:dev_sse into d1a703144a5615bbf7df0375bb1194047dfff5d0 on robgolding:master.

Coverage increased (+0.07%) to 98.529% when pulling 60a0e546448861fab84396c80e50232aea0585f2 on lydiaauch:dev_sse into d1a703144a5615bbf7df0375bb1194047dfff5d0 on robgolding:master.

Thank you @lydiaauch! This looks fantastic 😄
Merging now. I'll bump the version and release shortly.


Make sure all cluster directories are always scanned, not only only t…

…hose that are in the selected config.

This should solve problems in building specific cluster, e.g., the groovy cluster, as experienced by @svenreimers?


Enhance documentation

Add information about this application

hi @iamuv2000 I am willing to help you with this. Can i proceed?


Python script for changing the wallpaper

hi its my first script in python feel free to review it 😊

#101

Don't forget to give a ⭐ & Happy coding!. 😊


[FEATURE] .srt string helper

Description
There should be a method in a static manager class which performs an operation where the input is a AsBroadcastScript.cs class and the output is a string formatted as an srt-file.
// Example output should be:

"1
00:00:00:00 --> 00:00:02:01
- Here's what I love most
about food and diet.

2
00:00:02:02 --> 00:00:06:14
We all eat several times a day,
and we're totally in charge

3
00:00:06:64 --> 00:00:09:10
of what goes on our plate
and what stays off."

Create a new class in a folder called Managers, classname SrtManager.cs. The class should be static. Create a method called GenerateSrtText(AsBroadcastScript script).
The above will serve as a good baseline for future implementations.. We will refactor this later once the Timecode-class has full support for ms-conversion.

This is deemed out of scope for the project. Moved to P


Add ability to import modules using absolute paths

Since working with relative paths (../../../) can be such a pain, we should investigate adding absolute import paths to the pipeline.
This Gist outlines four possibilities:

The Symlink
The Global
The Module
The Environment

In my experience I've only used the NODE_PATH= environment variable (which worked as expected), but in a complex environment like Force (and possibly Heroku, though this article seems to suggest it's fine) there could be problems to investigate.
The Browserify Handbook also outlines some options.
Additionally, since we're using Babel, it would be worth investigating babel-plugin-module-resolver which compiles down paths into a safe format via a config step. We're currently using this in in Prediction.
cc @craigspaeth

I like the idea of using NODE_PATH—especially if it integrates nicely with Heroku config and dot env files.
When I've read about this I noticed a lot of people will only specify one folder as blessed to be added to the path like /lib. I think there's a valid argument there about how using absolute paths across the board makes it less obvious when something might be breaking its "modularity". For instance, I'd like to encourage people to design sub-apps in Force as self-contained slices of functionality behind a set of routes. One should be able to drop apps simply by deleting the folder and taking the line out where it's mounted (e.g. not requiring into other apps). A tangible example of taking advantage of this design is when redesigning a page we'll often "fork" the app (e.g. apps/artist vs. apps/artist2) and delete the loser of the AB test. So maybe it's worth considering absolute requires for project-wide modules like components/lib/models/collections and relative otherwise.
Tangentially related, I'd also love to hear others thoughts about the folder structure now with Force & MG merged. I think ideally we'd decide to more closely integrate/flatten /desktop and /mobile folders now that the hard merger part is over, and if we change NODE_PATH it could be a good opportunity to switch things up. e.g. if we wanted to bless the lib folder as another "node modules" folder we could consider a structure like...
- /apps
  - /artwork
    - /mobile # Pragmatic step towards co-locating MG's app code with Force
    - index.js # Mounts the mobile folder above as a sub-app
- /lib
  - /components
  - /models
  - /collections
  - /middleware
  - /etc

Thanks for starting this thread @damassi !!

Totally agree about encouraging boundaries for modularity. I think that when a project gets to the size of this one you start to see less interaction between files within a single "app" while heavy interaction outside of it. Looking at apps/auction/index.coffee this jumps right out:
CurrentUser = require '../../../models/current_user.coffee'
Artworks = require '../../../collections/artworks.coffee'
AuctionArtworksView = require '../../../components/auction_artworks/view.coffee'
mediator = require '../../../lib/mediator.coffee'
setupClocks = require './clocks.coffee'
myActiveBidsTemplate = -> require('../templates/my_active_bids.jade') arguments..
EmailRegistrationView = require './email_registration.coffee'
...

A quick glance shows us that everything after ./ is within our route, and everything below it is outside of it. Since most of the models and shared components and so on are global, introducing NODE_PATH (or something similar) would immediately clean this up and actually make the code more portable and modular since paths unconcerned with the specific route (which generally don't move that far laterally) never have to be touched, regardless of where the module / route / whatever  might move or go.
Of course, code that's nested into folders inside of a route will eventually have to refer to other code in the route so ../ style paths might still appear, but since the distance is (typically) only one or two levels deep its not so bad. As convention takes hold seeing the ../ would then be an indicator of a much narrower import scope -- it concerns only the files that are specific to our route.
All that said, I think one of my complaints with relative paths is that it's so rare when looking at the practices of other langs, and thus becomes more of a philosophical question about whats the best way to do things. Since Node's way is something of an outlier, it makes me wonder why other langs typically import off a full package path, and why it's rare to see relative (reverse) imports. The question comes down to which provides more value, because having explicit, absolute path imports provides a lot of metadata about whats going on and might be less error prone than keeping the paths masked (but more portable) via ../foo.
Also, +1 for colocating mobile stuff alongside desktop!

Another thing I'd like to add about relative path difficulties is project exploration and especially refactoring -- it's difficult to see whats been used where when each path can potentially be different.

When evaluating the various options, please also consider how well editors will be able to open imported files (i.e. that they can reason about the paths) as well as a language like TypeScript (which basically removes the Babel option), as I’d like 1 solution for all cases.

Spoke to an MS employee, they’re using the following combination:

TS/Intellisense works by specifying the baseUrl option in the TS config.
babel-plugin-module-resolver to actually rewrite the paths in the resulting JS.

So the babel plugin would work after all.

@alloy - yup, I feel the babel plugin is best as well since it will work seamlessly across the client / server boundary of the stack.
@craigspaeth - Could we go ahead and give this a spin in Auctions 2? Thoughts?


Update security notice URL

The anchor is lower-case

https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions


Release log after exception so that old log name can be used again.

Hi. I had problems when there's an error in VSS access and unhandled exception occurs. Log file remains open and I had to change the file name in UI to be able to continue.
This change just releases the log file after unhandled exception.

It looks like you must have changed the line endings, as the diff is the whole file. Please fix this and I'll take a look.

Yes, sorry. I'll fix it when I get back to work.

Replaced with fixed version


Feature - Find Tag command

This task will cover:

[ ] Searching contacts using a given Tag
[ ] Adding a command to find contacts by a given Tag

Reopened as added wrong issue for #144


install with pip

is there a way to install this library with pip?
I feel many could be confused between this and the standard python logger library.

Hi, the package is not registered on pypi if that's your question. If needed, you can still use a pip install -e . in a local version of the repository (e.g. after a git clone).
This is a small and simple package I've found to be useful for my research. It is meant to be a small codebase easy to tweak and adapt, not a large and widespread library. I understand it can be confused for the python logging library, but I don't consider this is to be a big issue given the use case. I'm open to suggestions though.

sure thing; and I apologize for opening an issue about this. There is something that pip can install that is just called 'logger', not 'logging', though I'm now unsure of what that thing is, since PyPI does not have a package that is simply called 'logger'.

No worries. The pypi package seems to be located here, it's just some configuration for the python standard logging.

If I could make a naive suggestion, not knowing much about use cases and such, perhaps the ambiguity could be resolved by renaming this package 'scientific-logger' or something like that?
But then again, perhaps this ambiguity is so rare it's not worthwhile haha.

Thanks for the suggestion, I'll consider it for future improvements.


Live Share Audio not responding

My VS Code Live Share audio extension is not working, whenever I try to start a call
I'm using macOS Monterey 12.1
Here's a small screenshot of the issue I'm facing whenever I try to start an audio call:-

I know you might suggest a solution like try clicking on the repair button but the repair button doesn't work too.

My network is really good


feat: Adds @register_theme decorator

Resolves one item in #3519
Fairly simple addition, but it is a nice convenience when you only use a single theme.
I also added an example that might be relevant to @joelostblom https://github.com/vega/altair/issues/3519#issuecomment-2267663552
The decorator also works well with type checkers:

Will probably need to resolve some conflicts after #3523 is merged

Very useful! Thanks for adding this and sorry that you had to wait a month for a review and keep this branch in sync ;)

Thanks for getting to this @binste and no worries on the timing; this is a mini-feature after all 😄


Remove unneeded substring

This was showing up as ~2% of all allocations in this trace: https://github.com/dotnet/sdk/pull/1432#issuecomment-316852561.

To make sure this wasn't causing additional overhead (as there's additional argument checking in the StringBuilder.Append(string, int, int) overload, I did a MicroBench mark:
[MemoryDiagnoser]
public class SubString
{
    public string Field = "This is a real long long argument";

    [Benchmark]
    public void StringBuilder_Append()
    {
        for (int i = 0; i < 10000; i++)
        {
            StringBuilder builder = new StringBuilder();

            for (int j = 0; j < 5; j++)
            {
                builder.Append(Field, 20, 11);
            }
        }
    }

    [Benchmark]
    public void StringBuilder_AppendViaSubString()
    {
        for (int i = 0; i < 10000; i++)
        {
            StringBuilder builder = new StringBuilder();

            for (int j = 0; j < 5; j++)
            {
                builder.Append(Field.Substring(20, 11));
            }
        }
    }
}

Method
Mean
Error
StdDev
Gen 0
Allocated

StringBuilder_Append
1.325 ms
0.0060 ms
0.0053 ms
589.8438
2.37 MB

StringBuilder_AppendViaSubString
1.896 ms
0.0115 ms
0.0108 ms
1019.5313
4.08 MB

Also did a quick pass and didn't see anymore easy ones like this.


[BEAM-13574] Filesystem abstraction Rename support

Build up the filesystem abstraction with Copy, Remove, and Rename support.
Implement these interfaces for the GCS, Local, and InMemory implementations as appropriate.
Unit test the utility functions that build from the abstraction, and backfill testing.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @jrmccluskey @youngoli
cc: @riteshghorse

PTAL I added tests for the in memory fs.

Run GoPortable PreCommit


i am new to laravel but interested in trying the  module package

i am new to laravel but interested in trying the  module package. I am confused with the step where should I need to pasted the below code
Autoloading
{
"autoload": {
"psr-4": {
"App\": "app/",
"Modules\": "Modules/"
}
}
}

In composer.json ;-)