docs: tutorial/example for ngclient Updater

EDIT: I think starting with out-of-the-box runnable example code is a better idea: see comment below (original issue follows:)
the updater module docs show how to write a  client in five lines of code but they also mention dependencies:
    * A webserver running on http://localhost:8000, serving TUF repository
      metadata at "/tuf-repo/" and targets at "/targets/"
    * Local metadata directory "~/tufclient/metadata/" is writable and contains
      a root metadata version for the remote repository
    * Download directory "~/tufclient/targets/" is writable

would be nice to have really easy to follow instructions that handle all of that...
The tutorial should live within the documentation but not in the ngclient module docs, just as a page on its own. Actually writing the tutorial is probably a "good first issue"?

I'm marking this good first issue even if it's not completely trivial: if you're interested, feel free to ask for a bit of advice.

The alternative to a tutorial might be to write a small example client that can just be executed, and short documentation for it -- the content is going to be roughly the same but I feel like an example is more useful?
Possible initial implementation using the tests repository data

client python code in examples/example-client.py
examples/metadata/root.json copied from tests repository
examples/targets/ for downloading targets
client could have a few features like "list targets", "download target", "clear cache" (to remove local targets and metadata except root)
examples/example-repository.py would just serve the test repository (tests/repository_data/repository/) over http

I would like to have a more featureful repo example (that would actually show the cool features of TUF) but this would be a start to show the client.

I'm picking up this issue.

I got into potential future plans in chat, I think they look good enough to include here -- most of it not as something to implement as part of initial example but as goals for future:
so these potential variations of example code setups would be pretty much the same for the client part, what changes is the remote end: where is the metadata (and targets) served and how are they generated. Obviously the decisions affect the client in the sense that some client features would only make sense in some of these cases.

local static server: python -m http.server -d tests/repository_data/repository/  -- use static metadata that already exists in python-tuf tests, serve it locally using http.server. This is a one-liner but does not show the cool things in TUF.
local dynamic server: Use Metadata API to generate metadata from scratch, write a small server to serve it locally: user would still run this on their own machine. This allows some nice possibilities like

updating the timestamp every 30 seconds so client sees things changing as they update
making the server react to file changes in a directory: copy a file into targets directory, and the server updates metadata automatically to include that target file
etc ...

Run an actual TUF repository somewhere on the internet. This is most complex but also feels much, much more real. In this case user does not need to start any servers on localhost: just run the client example and watch it securely download files from the internet

this requires infrastructure to run the timestamp and snapshot processes (see github workflows in https://github.com/jku/tuf-repo-test for a taste)
this also requires some content flow: something needs to be added into the metadata once in a while -- it could be python-tuf release files during the release process but could be something completely different, possibly automated, as well
note that actual target files don't need to be included in the repository: e.g. python-tuf release files can be pypi.org hosted, we just serve the metadata for the TUF repo
this might be doable with github actions in python-tuf repo deploying metadata to e.g. python-tuf.github.io (which is currently unused)

Before submitting the PR, I am sending a design proposal/draft for local static server
The idea is to start with a simple document describing how to run an example of a client code using the ngclient and the static metadata from tests/repository_data/repository/.
The document will have three basic steps.

Run the HTTP Server using the tests data repository
How to initialize the client to prepare the client infrastructure
Download files to show different scenarios
In that step, we will highlight the TUF/ngclient behaviors.

The client code example:

The example will be stored inside the examples/ in the root folder. The reason is that some people check examples, even before the documentation.

The idea is a commented code, including references to the API and/or the Specification.

The initial name suggested will be client_example as a small Python Command Line Interface (CLI) executable/script.

The metadata repository directory will be in $HOME/.local/share/<app-name-here>/
The download folder directory will be in download (local directory)

The example will be a simple CLI with initial two parameters:

--init
This parameter will create the client's infrastructure, the metadata folder (including the initial root.json), and the download folder.
Running this example will be explicit about all the steps. The CLI will print the messages.

./client_example --init
[INFO] metadata folder [<path>] created.
[INFO] Bootstrap 'root.json' with a trusted source.
[INFO] Download folder  ['downloads'] created.
[INFO] Client initialization finished.

The second scenario is the client already initialized, but the messages above will be the same.

--download <filename>.
Along with the how-to documentation, the instruction will be to download different targets.
The goal is to demonstrate flow, the terminology used in the framework, using the steps messages (printing as log), such as:

./client_example --download file.txt
[INFO] Updating metadata from remote.
[INFO] Getting target info.
[INFO] Checking if the target <name> is cached.
[INFO] Downloading target <name> to download <path name>.
[INFO] File downloaded successfully.

on the CLI design: looks good. Download could be a sub-command as well (so download file.txt instead of --download file.txt), then combining with --init would maybe look better: ./client_example --init download file1.txt.
I'm mostly thinking about if you add more commands later (like list-files) then if they are both options it looks like you could combine them (--list-files --download file1.txt) which is probably not what we want: if they are commands it's clear you should only give one command at a time.
One thing we didn't discuss is how do we handle the server startup in this first iteration? the client example could of course start a new local http server in the background each time it runs... but maybe the documentation just includes advice to run python3 -m http.server ... in another terminal? In the latter case you just need to make the port selection somehow trivial -- I don't think we want the user copy pasting port numbers... maybe hard code a specific port? any better ideas?

That said, maybe we should not provide an example of --init at all (I'm not 100% sure if you meant downloading the initial root.json over http in a Trust-On-First-Use model)? Maybe making a copy of the root.json in the examples directory and bootstrapping the local metadata directory from there (in other words copying the file to local metadata dir) is a more responsible example?

on the CLI design: looks good. Download could be a sub-command as well (so download file.txt instead of --download file.txt), then combining with --init would maybe look better: ./client_example --init download file1.txt.

I think using sub-commands also looks good, and it is something usual now, also helps to extend.
IMO, having --init as a parameter in combination can be valuable, but is it instructive? I will add but not in the documentation explicitly; however, we can change it easily in the doc only.

I'm mostly thinking about if you add more commands later (like list-files) then if they are both options it looks like you could combine them (--list-files --download file1.txt) which is probably not what we want: if they are commands it's clear you should only give one command at a time.
One thing we didn't discuss is how do we handle the server startup in this first iteration? the client example could of course start a new local http server in the background each time it runs... but maybe the documentation just includes advice to run python3 -m http.server ... in another terminal? In the latter case you just need to make the port selection somehow trivial -- I don't think we want the user copy pasting port numbers... maybe hard code a specific port? any better ideas?

I am planning to have it in the documentation as a kind of how-to...
The draft I am working on
Preparing
To have the example working in your machine, clone the python-tuf in your system.
$ git clone git@github.com:theupdateframework/python-tuf.git

Repository
As this example demonstrates how to use the python-tuf to build a client application, the repository will use static files.
The static files are available in the python-tuf repository, same as this
The static repository files are in
tests/repository_data/repository.
Run the repository using the Python3 built-in http module, and keep this
session running.
$ python3 -m http.server -d tests/repository_data/repository
Serving HTTP on :: port 8000 (http://[::]:8000/) ...

(...)

That said, maybe we should not provide an example of --init at all (I'm not 100% sure if you meant downloading the initial root.json over http in a Trust-On-First-Use model)? Maybe making a copy of the root.json in the examples directory and bootstrapping the local metadata directory from there (in other words copying the file to local metadata dir) is a more responsible example?
If we do that, then I think --init can be implicit: if the root.json does not exist in the correct dir, copy it over

Yes, I was thinking about it this morning when I was writing the code.
Bootstrapping from a copy stored with the example is best in the sense of giving the best practice IMO.

The initial version for the client example was merged by PR #1675. I will add here some next steps commented in this PR.
I think we can continue the discussion here and create new issues (smaller issues) to track them.
From this comment https://github.com/theupdateframework/python-tuf/pull/1675#pullrequestreview-818668044 (thanks @jku for the insights)

https://github.com/theupdateframework/python-tuf/issues/1716
https://github.com/theupdateframework/python-tuf/issues/1717
support other server locations than hard-coded localhost:8000
support boostrapping root.json for other servers (like the repository generated by the repository example) somehow
document the application decisions better (as an example "why is is local metadata stored in $HOME/.local/share/", "why - - not download first root.json over http?"). I didn't do this in the review as I think it's a tricky balance: too much detail means no-one will read it...
start thinking about how to use examples to show that TUF actually works, and prevents attacks. This might be more related to repository examples but ...

Thanks. I'd like to close this one to mark that the dependency for 1694 is handled...
Maybe it's ok that the main missing features are issues and remaining  ideas are searchable here... or you could file another issue for the "support other servers" issue.
Closing this one, feel free to file new issues for the other ideas


fix dead lock detection for stand alone mode

Close grpc connection when rebuild stream, otherwise the leaked socket will finally result in too many open files error.
Set role when server starts in stand-alone mode.
Report leader peer to PD when server start.
PTAL @coocood @cfzjywxk

LGTM

LGTM


Suggested Feature

My husband mentioned to me how cool it would be if other people using the deadname remover could also see the name changes and have some way to recognize that someone else's name was replaced so they didnt out them, or maybe people could add eachother to a list of people who could see the replaced name. I dont know how easy this would be to implement but it would offer some more degrees of "out" you know? anyways i really appreciate any time you took to read this. thank you so much!

Hello! That would be a great thing but I think it's too hard to do. It would require a lot of things, this extension is for now very simple. I don't think it will be done.
(I'm writing this but I'm not related to this extension and its author)

Thats alright, we thought that may be the case, it was just a really neat
idea that if we didnt share no one would know you know? Sometimes ideas
dont work out quite that simple
On Thu, Aug 20, 2020, 11:38 AM aopom<EMAIL_ADDRESS>wrote:

Hello! That would be a great thing but I think it's too hard to do. It
would require a lot of things, this extension is for now very simple. I
don't think it will be done.
(I'm writing this but I'm not related to this extension and its author)
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/WillHayCode/Deadname-Remover/issues/47#issuecomment-677773660,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQUYUC4S2SMZ4CO6BS6EQR3SBVGQRANCNFSM4QCTTI6Q
.


blacklist bl.eamilbasuara.org is dead and should be removed from list

https://www.dnsbl.com/2019/10/status-of-blemailbasuraorg-dead.html
Status of bl.emailbasura.org: DEAD
The DNSBL Email Basura is no more. Email Basura ("Trash" in Spanish) appears to have been online since at least 2004, according to the Internet Archive. This anti-spam blacklist's DNSBL zone was "bl.emailbasura.org."
The domain emailbasura.org seems to have expired and been purchased by a domain speculator. The domain has wildcard DNS entries, meaning that any use of the old DNSBL zone in your email server may result in your server blocking all inbound mail. You don't want that! Remove the DNSBL zone "bl.emailbasura.org" from your mail server configuration as soon as possible.
Posted by Al Iverson on Wednesday, October 30, 2019
Labels: bl.emailbasura.org, dead dnsbls, dnsbl

Tnx released a new version!


updated to new ini, but still not able to connect session

When i check the configuration after updating the ini... I have everything green and good to go..

But when i connect from another pc i get the following error,
account restrictions are preventing this user from signing in
any help on this?

Did you ever used RDP before?
The user you signing in must be in "Administrators" group, or "Remote Desktop Users" (if you don't want to make it admin).

i had been using the rdp wrapper on this machine, only after the windows update from yesterday that it stopped working, i waited for the new ini.. went ahead and updated... and got this error... let me verify user permission but i dont recall modifying them...

ok... found the problem, apparently it's something related to the crentials saved on the client pc... i tried connecting via the pc's ip... and guess what it connected... tried again with the saved pc profile and credentials.. and same error... deleted remote desktop profile and recreated it .. using the computer name... and voila... it's connecting.... weird error.. but.. it's working... thanks and keep up the good work...

This can happen when high security group policy is set for Terminal Services Client, that doesn't allow to save credentials.
However Windows Update should not modify the group policies....

Had same issue after update of windows yesterday but deleting user profiles was not an option.
After update when i tried to login i seen message about restriction not allowing me to login.
I found a simple solution:
remove all users from "Remote Desktop users" group and save.
add them back again to this group and save.
And all works again great, only one thing - connection certificate was changed and i needed to accept it once again but not sure is it related to update of windows or just to removing and re-adding to group.
Hope it will help those who can't loose all data stored in user profiles.
Good piece of software, thank you.

most probably .. since.. i did not have that problem, until the update..
On Mon, Sep 19, 2016 at 7:56 AM, Magik1971<EMAIL_ADDRESS>wrote:

Had same issue after update of windows yesterday but deleting user
profiles was not an option.
After update when i tried to login i seen message about restriction not
allowing me to login.
I found a simple solution:
remove all users from "Remote Desktop users" group and save.
add them back again to this group and save.
And all works again great, only one thing - connection certificate was
changed and i needed to accept it once again but not sure is it related to
update of windows or just to removing and re-adding to group.
Hope it will help those who can't loose all data stored in user profiles.
Good piece of software, thank you.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/stascorp/rdpwrap/issues/161#issuecomment-247985472,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AVMS3i7d3dOdZsowkm4idBfzJkJjVuA2ks5qroaLgaJpZM4J-FGG
.

--
David Mendoza

I recommend you restore Windows Firewall settings to default.
Press the key on the Windows.
Type Windows Firewall, after click restore defaults.
Return to the Default settings of windows (All Firewall permissions settings will be lost).
Go to > allow a program or feature through Windows Firewall.
Enable file and printer sharing
-Network discovery
-SSTP (Secure Socket Tunneling Protocol)
Then in Windows Services Check the following services
-DNS Client,
-Discovery Resource Publication,
-SSDP Discovery Discovery
-UPnP Device Host
If any of them is disabled put on automatic and start the service.
If it persists, I recommend you try to reinstall the system.

I recommend you restore Windows Firewall settings to default.
Press the key on the Windows.
Type Windows Firewall, after click restore defaults.
Return to the Default settings of windows (All Firewall permissions settings will be lost).
Go to > allow a program or feature through Windows Firewall.
Enable file and printer sharing
-Network discovery
-SSTP (Secure Socket Tunneling Protocol)
Then in Windows Services Check the following services
-DNS Client,
-Discovery Resource Publication,
-SSDP Discovery Discovery
-UPnP Device Host
If any of them is disabled put on automatic and start the service.
If it persists, I recommend you try to reinstall the system.

Closed as solved. If not – reopen it


[ADD] Bioconda install

Added recipe for bioconda, see bioconda-recipes PR #45189

See: https://anaconda.org/bioconda/massdash


Do anyone tried to convert trained .h5 model to Tensorflow Lite or Tensorflow Serving?

Do anyone tried to convert trained .h5 model to Tensorflow Lite or Tensorflow Serving?
I got some difficulties due to some python functions such as 'yolo_loss' and 'yolo_head'.

The last days I tried to convert a tiny model to Tensorflow Lite.
In order to overcome the issue with 'yolo_loss' and 'yolo_head', I used the model created by tiny_yolo_body (and loaded my weights).
Because of over issues, the input had to be fixed (in my case to (416, 416, 3)).
Again some error occured and forced me to install tf-nightly (without gpu) in a different v.-env. . Now I'm stuck with the error F tensorflow/lite/toco/graph_transformations/propagate_fixed_sizes.cc:117] Check failed: dim_x == dim_y (16 vs. 32)Dimensions must match . I have no idea how to fix it.

I tried to create the model like this:
yolo_model = YOLO(**vars(FLAGS)).yolo_model
yolo_model.summary()
And convert the model using the following code:
def save_model_for_production(model, version, path='prod_models'):
tf.keras.backend.set_learning_phase(1)
if not os.path.exists(path):
os.mkdir(path)
export_path = os.path.join(
tf.compat.as_bytes(path),
tf.compat.as_bytes(version))
builder = tf.saved_model.builder.SavedModelBuilder(export_path)
model_input = tf.saved_model.utils.build_tensor_info(model.input)
model_output = tf.saved_model.utils.build_tensor_info(model.output)

prediction_signature = (
    tf.saved_model.signature_def_utils.build_signature_def(
        inputs={'inputs': model_input},
        outputs={'output': model_output},
        method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME))

with tf.keras.backend.get_session() as sess:
    builder.add_meta_graph_and_variables(
        sess=sess, tags=[tf.saved_model.tag_constants.SERVING],
        signature_def_map={
            'predict':
                prediction_signature,
        })

    builder.save()

However, I got the following error:
model_output = tf.saved_model.utils.build_tensor_info(model.output)
Exception has occurred: AttributeError
'list' object has no attribute 'dtype'
File "D:\tensorflow\keras-yolo3-master\model_conversion.py", line 22, in save_model_for_production
model_output = tf.saved_model.utils.build_tensor_info(model.output)
File "D:\tensorflow\keras-yolo3-master\model_conversion.py", line 90, in
save_model_for_production(yolo_model, "1", export_path)

The error is a result of the line model_output = tf.saved_model.utils.build_tensor_info(model.output). The YOLO models have multiple outputs and you need to handle each output separately (and adjust the signature ('output1': model_output[0], 'output2': model_output[1],  ...)). I tried the modified code and faced other errors which I'm not able to fix (tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value conv2d_9/kernel          [[node conv2d_9/kernel (defined at C:\venv4\lib\site-packages\keras\backend\tensorflow_backend.py:402) ]]          [[batch_normalization_9/beta/_161]]).

I tried Flask today, which is much more easier to setup.

I found that there is no 'Attempting to use uninitialized' error if I call the following function first:
sess.run(tf.global_variables_initializer())

I found that there is no 'Attempting to use uninitialized' error if I call the following function first:
sess.run(tf.global_variables_initializer())

@marco8chong
So you were able to get rid of all error messages? Can you please provide the script you finally used?
I am not able to understand this:

I tried to create the model like this:
yolo_model = YOLO(**vars(FLAGS)).yolo_model
yolo_model.summary()

What are the FLAGS and how did you provide the already trained data from the h5 file?

I added in train.py a model.save(log_dir + 'trained_model_final.h5') next to the model.save_weights statement. I moved this result to model_data/tf.h5
I try to load the dataset to be able to convert it with your script snipped as follows:
# https://github.com/qqwweee/keras-yolo3/issues/344

import tensorflow as tf
import os
from keras.models import load_model

#https://github.com/qqwweee/keras-yolo3/issues/48#issuecomment-457440948
from yolo3.model import yolo_head

def _main():
    path = "model_data/"

    model = load_model(os.path.join(path, 'tf.h5'), {'yolo_head': yolo_head})
    tf.keras.backend.set_learning_phase(1)
    if not os.path.exists(path):
        os.mkdir(path)
    export_path = os.path.join(tf.compat.as_bytes(path))
    builder = tf.saved_model.builder.SavedModelBuilder(export_path)

    model_input = tf.saved_model.utils.build_tensor_info(model.input)
    model_output = tf.saved_model.utils.build_tensor_info(model.output)

    prediction_signature = (
        tf.saved_model.signature_def_utils.build_signature_def(
            inputs={'inputs': model_input},
            outputs={'output': model_output},
            method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME))

    with tf.keras.backend.get_session() as sess:
        sess.run(tf.global_variables_initializer())
        builder.add_meta_graph_and_variables(
            sess=sess, tags=[tf.saved_model.tag_constants.SERVING],
            signature_def_map={
                'predict':
                    prediction_signature,
            })

        builder.save()

if __name__ == '__main__':
    _main()

But i get the error:
  File "<PathToTheProject>keras-yolo3/yolo3/model.py", line 376, in yolo_loss
    anchors[anchor_mask[l]], num_classes, input_shape, calc_loss=True)
TypeError: list indices must be integers or slices, not list

Can anyone guide me the last steps here?

I tried to convert the model to Tensorflow Serving like this, the code can run but obviously I missed some parts too.
`
import sys
import argparse
import tensorflow as tf
import os
from yolo import YOLO
from keras import backend as K
from keras.models import Model
tf.logging.set_verbosity(tf.logging.INFO)
FLAGS = None
def save_model_for_production(model, version, path='prod_models'):
tf.keras.backend.set_learning_phase(1)
if not os.path.exists(path):
os.mkdir(path)
export_path = os.path.join(
tf.compat.as_bytes(path),
tf.compat.as_bytes(version))
builder = tf.saved_model.builder.SavedModelBuilder(export_path)
model_input = tf.saved_model.utils.build_tensor_info(model.input)
model_output0 = tf.saved_model.utils.build_tensor_info(model.output[0])
model_output1 = tf.saved_model.utils.build_tensor_info(model.output[1])
model_output2 = tf.saved_model.utils.build_tensor_info(model.output[2])

prediction_signature = (
    tf.saved_model.signature_def_utils.build_signature_def(
        inputs={tf.saved_model.signature_constants.PREDICT_INPUTS: model_input},
        outputs={tf.saved_model.signature_constants.PREDICT_OUTPUTS+'0': model_output0,
                 tf.saved_model.signature_constants.PREDICT_OUTPUTS+'1': model_output1,
                 tf.saved_model.signature_constants.PREDICT_OUTPUTS+'2': model_output2},
        method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME))

with tf.keras.backend.get_session() as sess:
    #
    sess.run(tf.global_variables_initializer())
    #
    builder.add_meta_graph_and_variables(
        sess=sess, tags=[tf.saved_model.tag_constants.SERVING],
        signature_def_map={
            'predict':
                prediction_signature,
        })

    builder.save()

if name == 'main':
# class YOLO defines the default value, so suppress any default here
parser = argparse.ArgumentParser(argument_default=argparse.SUPPRESS)
'''
Command line options
'''
parser.add_argument(
'--model', type=str,
help='path to model weight file, default ' + YOLO.get_defaults("model_path")
)
parser.add_argument(
    '--anchors', type=str,
    help='path to anchor definitions, default ' + YOLO.get_defaults("anchors_path")
)

parser.add_argument(
    '--classes', type=str,
    help='path to class definitions, default ' + YOLO.get_defaults("classes_path")
)

parser.add_argument(
    '--gpu_num', type=int,
    help='Number of GPU to use, default ' + str(YOLO.get_defaults("gpu_num"))
)

parser.add_argument(
    '--image', default=False, action="store_true",
    help='Image detection mode, will ignore all positional arguments'
)
'''
Command line positional arguments -- for video detection mode
'''
parser.add_argument(
    "--input", nargs='?', type=str,required=False,default='./path2your_video',
    help = "Video input path"
)

parser.add_argument(
    "--output", nargs='?', type=str, default="",
    help = "[Optional] Video output path"
)

FLAGS = parser.parse_args()

yolo = YOLO(**vars(FLAGS))
yolo_model = yolo.yolo_model
yolo_model.summary()

export_path = "tf-model-serving"
save_model_for_production(yolo_model, "1", export_path)

`

@marco8chong Thanks a lot for the code snipped.
That helped me to see how i load the trained keras weights as a model. I was able to run your script. In case of the yolo-tiny model, i got the problem that model.output[2] was out of range. I removed this output accordingly. Afterwards i was able to use your script to generate the pb, data and index file. Nevertheless i was not able to convert them easily into a tflite file.
I found this yolo3 project, which exports tflite directly: https://github.com/benjamintanweihao/YOLOv3/blob/master/tflite.py.
Merging your dataset-load with his conversion step produced me finally a ftlite file :+1:
I have to clean it up a lot and check if the generated file does really do the job, but i think this could be the solution.

Here you get the script snipped i used for conversion. This must be placed as an python file directly in the root directory of the keras-yolo3 project:
# https://github.com/qqwweee/keras-yolo3/issues/344
from yolo import YOLO

import argparse
from tensorflow.contrib.lite.python import lite

def _main():
    # class YOLO defines the default value, so suppress any default here
    parser = argparse.ArgumentParser(argument_default=argparse.SUPPRESS)
    '''
    Command line options
    '''
    parser.add_argument(
        '--model', type=str,
        help='path to model weight file, default ' + YOLO.get_defaults("model_path")
    )

    parser.add_argument(
        '--anchors', type=str,
        help='path to anchor definitions, default ' + YOLO.get_defaults("anchors_path")
    )

    parser.add_argument(
        '--classes', type=str,
        help='path to class definitions, default ' + YOLO.get_defaults("classes_path")
    )

    parser.add_argument(
        '--gpu_num', type=int,
        help='Number of GPU to use, default ' + str(YOLO.get_defaults("gpu_num"))
    )

    parser.add_argument(
        '--image', default=False, action="store_true",
        help='Image detection mode, will ignore all positional arguments'
    )
    '''
    Command line positional arguments -- for video detection mode
    '''
    parser.add_argument(
        "--input", nargs='?', type=str, required=False, default='./path2your_video',
        help="Video input path"
    )

    parser.add_argument(
        "--output", nargs='?', type=str, default="",
        help="[Optional] Video output path"
    )

    FLAGS = parser.parse_args()

    yolo = YOLO(**vars(FLAGS))
    yolo_model = yolo.yolo_model
    yolo_model.summary()

    keras_model_path = "model_data/tf.h5"
    yolo_model.save(keras_model_path)
    converter = lite.TFLiteConverter.from_keras_model_file(
        keras_model_path, input_shapes={'input_1': [1, YOLO.get_defaults("model_image_size")[0],
                                                    YOLO.get_defaults("model_image_size")[1], 3]})

    converter.post_training_quantize = True
    tflite_model = converter.convert()
    open("model_data/tf.tflite", "wb").write(tflite_model)

if __name__ == '__main__':
    _main()

Now my problem is the missing Yolo Head

Now my problem is the missing Yolo Head

This might help you:
https://github.com/qqwweee/keras-yolo3/issues/349#issuecomment-486345613
Nevertheless, the script i posted did not need any further import, when placed as a sibling to yolo.py

Actually my problem is that the outputs of the converted model are 3 convolution layers, and I have no idea about how to create detected boxes, scores and classes.

The output is described in the two following papers:
https://pjreddie.com/media/files/papers/YOLO9000.pdf
https://pjreddie.com/media/files/papers/YOLOv3.pdf
The yolo_head function in model.py is implemented as explained in the papers.
Here is an example of an implementation in Kotlin.
Here is my implementation (for Tiny Yolo, so there are a 2 outputs, you need 3) in Java:
    float[][][][] output1 = new float[1][outputSizes[0]][outputSizes[0]][(numberOfClasses + 5) * 3];
    float[][][][] output2 = new float[1][outputSizes[1]][outputSizes[1]][(numberOfClasses + 5) * 3];

    outputs.put(0, output1);
    outputs.put(1, output2);

    List<Detection> results = new LinkedList<>();

    tf.runForMultipleInputsOutputs(inputs, outputs);

    float[][][][] active = output1;

    int offset;

    float confidence, x, y, w, h,
            maxClass;

    int objectId;

    float[] confidenceClasses = new float[numberOfClasses];

    /*
     * Based on: J. Redmon and A. Farhadi. Yolov3: An incremental improvement. arXiv, 2018. 4
     */
    for(int i=0;i<2;++i) {
        for(int j=0;j<outputSizes[i];++j)
            for(int k=0;k<outputSizes[i];++k)
                for(int l=0;l<3;++l) {
                    offset = (numberOfClasses + 5) * l;

                    confidence = MathFunctions.sigmoid(active[0][j][k][offset + 4]);

                    if(confidence < threshold)
                        continue;

                    if(numberOfClasses == 1) {
                        maxClass = 1;

                        objectId = 0;
                    } else {
                        for(int m=0;m<numberOfClasses;++m)
                            confidenceClasses[m] = active[0][j][k][offset + 5 + m];

                        objectId = determineClass(confidenceClasses);

                        maxClass = confidenceClasses[objectId];
                    }

                    confidence *= maxClass;

                    if(confidence > threshold) {
                        x = (k + MathFunctions.sigmoid(active[0][j][k][offset])) * (float)size / outputSizes[i];
                        y = (j + MathFunctions.sigmoid(active[0][j][k][offset + 1])) * (float)size / outputSizes[i];

                        w = (float)Math.exp(active[0][j][k][offset + 2]) * anchors[(i * 3 * 2) + (l * 2)];
                        h = (float)Math.exp(active[0][j][k][offset + 3]) * anchors[(i * 3 * 2) + (l * 2) + 1];

                        results.add(new Detection(objectId, confidence,
                                new RectF(Math.max(0, x - w / 2), Math.max(0, y - h / 2), Math.min(size - 1, x + w / 2), Math.min(size - 1, y + h / 2))));
                    }
                }

        active = output2;
    }

    return pooling.execute(results);

The most inner loop is for the anchors. pooling is a custom lambda function that will execute some kind of ROI pooling.

This discussion is very helpful! Thank you for posting these scripts. Can you please confirm if you were able to get the TFLite version of YoloV3 fully working on Android? I am a bit confused by the code snippets because I see that the conversion script creates a Quantized model (8 bit) but the Java inference code uses the Float datatype.... Also, I am very curious what kind of inference timing you get on an actual mobile phone?

@dkashkin
I am not able to get YOLOv3 works on Android, but using other object detection models from Google is more straightforward.

This discussion is very helpful! Thank you for posting these scripts. Can you please confirm if you were able to get the TFLite version of YoloV3 fully working on Android? I am a bit confused by the code snippets because I see that the conversion script creates a Quantized model (8 bit) but the Java inference code uses the Float datatype.... Also, I am very curious what kind of inference timing you get on an actual mobile phone?

I'm sorry for the confusion. The code snippets were supposed to serve as example of extracting the bounding boxes of the output tensors, I did not look at the python snippets and missed the quantization.
My model is a non-quantized tiny yolov3 tflite model, which I did not obtain through the scripts here (I found a way to convert a tiny yolov3 darknet model to tflite, training in darknet is much more comfortable for me (convert cfg + weights with https://github.com/xiaochus/YOLOv3 to h5 -> convert h5 to pb -> convert pb to tflite)). The inference time is about 800ms (I forgot the exact time) on a Asus Zenfone 3 Zoom S (the Unihertz Atom needs a little bit more time).

Thanks @BenSchl I've been testing performance of quantized vs non-quantized models on Android with pretty surprising results (https://stackoverflow.com/questions/55958129/tensorflow-lite-quantization-fails-to-improve-inference-latency) Anyways my next goal is to test the Tiny YOLOv3. I already converted the Darknet model to Tensorflow, but struggling to figure out how to freeze the graph and convert it to TFLite. Can you please share the code that you used to get from Tensorflow to Tensorflow Lite?

The code I use is the code snippet I posted above (a link to the YOLOv3 paper is there, also).
The output tensors have the size N x N x [A * (4 + 1 + C)], where N x N is the size of the "grid" (the dimension clusters), A is the number of anchors for that tensor and C is the number of classes. The first 4 values in each offset is for the bound box prediction, the following value is the object confidence (therefore the 4 + 1).
In Tiny  YOLOv3, the clusters normally have the dimensions 13x13 and 26x26 (Tiny YOLOv3 has two output tensors). The Anchors for the 13x13 "grid" are often (81,82), (135,169), (344,319) and (10,14), (23,27), (37,58) for the 26x26 "grid" (Therefore the inner loop for(int l=0;l<3;++l) in the code snippet. The most outer loop is for the two output tensors.)
To answer you previous question: To convert Tiny YOLOv3 to tflite (using https://github.com/xiaochus/YOLOv3) I used something like
import tensorflow as tf
from tensorflow.python.framework import graph_util
from tensorflow.python.framework import graph_io

with tf.Session() as sess:
    tf.keras.backend.set_learning_phase(0)
    model = tf.keras.models.load_model('yolo.h5')
    constant_graph = graph_util.convert_variables_to_constants(
        sess,
        sess.graph.as_graph_def(),
        ["conv2d_10/BiasAdd", "conv2d_13/BiasAdd"])

graph_io.write_graph(constant_graph, "saved_model_pb",
                    "yolo.pb", as_text=False)

to convert the h5 model to pb and something like (but now not the gpu version of tensorflow)
import tensorflow as tf
from tensorflow.python.platform import gfile
from tensorflow import lite

with tf.Session() as sess:
    tf.keras.backend.set_learning_phase(0)
    with gfile.FastGFile('saved_model_pb/yolo.pb', 'rb') as f:
        graph_def = tf.GraphDef()
        graph_def.ParseFromString(f.read())
        g_in = tf.import_graph_def(graph_def)
        converter = lite.TFLiteConverter.from_session(sess, [sess.graph.get_tensor_by_name('import/input_1:0')], [sess.graph.get_tensor_by_name('import/conv2d_10/BiasAdd:0'), sess.graph.get_tensor_by_name('import/conv2d_13/BiasAdd:0')])
        tflite_model = converter.convert()
        open("yolo.tflite", "wb").write(tflite_model)

to convert the pb model to tflite.

Thank you very much @BenSchl I was able to convert Tiny Yolo V3 to TFLite and test it out on a few android phones. I am getting 770ms inference on Pixel2 (using float 416x416 model). Needless to say, this is too slow to support yolo's original "real time" claim :) This detector is much slower than SSD-based googlenet v2 although a bit higher resolution. Anyways I hugely appreciate your help and I owe you a large amount of beer :)

@marco8chong I have the same problem with  'yolo_loss' and 'yolo_head', Do you have saved it?

I finished it here, please try: https://github.com/peace195/tensorflow-lite-yolo-v3

@marco8chong 你flask是如何进行改变的（web），我需要你的帮助，谢谢

我現在查閱不到自己以前的 Code, 用 Flask 不用轉換 Model 格式, 當自己用 PC 版本便可, 方法大概如下:
https://blog.techbridge.cc/2018/11/01/python-flask-keras-image-predict-api/

@marco8chong 当时您在使用flask时有遇到graph的问题吗？我现在这个点上很困扰。我把这个项目整合到flask里面，flask的线程机制和tensorflow发生了矛盾导致graph发生报错。

沒有, Tensorflow 可以處理到線程的問題, 小心跟著網上的示範做應該是可以的
https://blog.victormeunier.com/posts/keras_multithread/

@marco8chong 谢谢您的回答，我的问题就在这里，在您给我的这篇文档中“This will load your model with the default graph and session from Tensorflow. If you try to do that in multiple threads, you'll have an error.”，这句话的原因是因为不同的线程加载不同的graph容易出现报错所以需要在每一个线程中设置thread_graph = Graph()设置为默认的graph，当你使用flask+keras时也会出现这个问题，第一遍的使用是正常的，第二遍的时候会显示graph问题，所以需要使用graph.as_default()这句函数，但是这个项目中使用的预测函数不是predict所以我很懊恼，不知道graph.as_default()这句话要加在哪里。

You can see at here. I tried it and success.
https://github.com/phanxuanduc1996/convert_yolo_weights

The error is a result of the line model_output = tf.saved_model.utils.build_tensor_info(model.output). The YOLO models have multiple outputs and you need to handle each output separately (and adjust the signature ('output1': model_output[0], 'output2': model_output[1],  ...)). I tried the modified code and faced other errors which I'm not able to fix (tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value conv2d_9/kernel [[node conv2d_9/kernel (defined at C:\venv4\lib\site-packages\keras\backend\tensorflow_backend.py:402) ]] [[batch_normalization_9/beta/_161]]).
Update:
If the model is compiled, the error changes to tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value batch_normalization_10/beta [[{{node batch_normalization_10/beta}}]] [[conv2d_6/kernel/_209]].
(Note: I'm using the tiny model).
Have you solved your problem?Now I have the same problem as you, and I don't know how to correct it

The error is a result of the line model_output = tf.saved_model.utils.build_tensor_info(model.output). The YOLO models have multiple outputs and you need to handle each output separately (and adjust the signature ('output1': model_output[0], 'output2': model_output[1],  ...)). I tried the modified code and faced other errors which I'm not able to fix (tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value conv2d_9/kernel [[node conv2d_9/kernel (defined at C:\venv4\lib\site-packages\keras\backend\tensorflow_backend.py:402) ]] [[batch_normalization_9/beta/_161]]).
Update:
If the model is compiled, the error changes to tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value batch_normalization_10/beta [[{{node batch_normalization_10/beta}}]] [[conv2d_6/kernel/_209]].
(Note: I'm using the tiny model).
@BenSchl
Hello，Have you solved your problem?Now I have the same problem as you, and I don't know how to correct it

I tried to convert Keras h5 format to TensorFlow serving, Please check this out.
https://github.com/LahiRumesh/YOLOv3-weights-converter


Query about order of timings

Love this library @mbdevpl . Was just playing around with some benchmarks for an article I plan to write on Python multiprocessing + numpy arrays, and I ran into some unintuitive behaviour:
    for _ in _TIME.measure_many("baseline", samples=n_frames):
        baseline_benchmark(np_arr)

    for _ in _TIME.measure_many("mp_queue", samples=n_frames):
        mp_queue_benchmark(np_arr, mp_queue)

    for _ in _TIME.measure_many("queue_module", samples=n_frames):
        queue_module_benchmark(np_arr, queue_module)

    timings.append(get_timings("baseline", n_frames))
    timings.append(get_timings("mp_queue", n_frames))
    timings.append(get_timings("queue_module", n_frames))

This code works. But rearranging the same code to:
    for _ in _TIME.measure_many("baseline", samples=n_frames):
        baseline_benchmark(np_arr)
    timings.append(get_timings("baseline", n_frames))
    for _ in _TIME.measure_many("mp_queue", samples=n_frames):
        mp_queue_benchmark(np_arr, mp_queue)
    timings.append(get_timings("mp_queue", n_frames))
    for _ in _TIME.measure_many("queue_module", samples=n_frames):
        queue_module_benchmark(np_arr, queue_module)
    timings.append(get_timings("queue_module", n_frames))

Returns
╰─❯ python array_benchmark/queue_benchmarking/queues.py
baseline: time: = 0.0008 +/- 0.0001 or FPS =<PHONE_NUMBER>9304002
Traceback (most recent call last):
  File "array_benchmark/queue_benchmarking/queues.py", line 100, in <module>
    DF = benchmark_queues()
  File "array_benchmark/queue_benchmarking/queues.py", line 83, in benchmark_queues
    timings.append(get_timings("mp_queue", n_frames))
  File "array_benchmark/queue_benchmarking/queues.py", line 61, in get_timings
    mean = f"{_TIME.summary[groupname]['mean']:.4f}"
KeyError: 'mp_queue'

Whereby I am using this function.
def get_timings(groupname, n_frames):
    """ Get a dictionary of the mean/std and FPS of the timing group.

    :param str groupname: name of the timing group
    :param int n_frames: number of timing repeats
    :return: mean/std and FPS of the timing group as a dictionary
    :rtype: dict
    """
    mean = f"{_TIME.summary[groupname]['mean']:.4f}"
    stddev = f"{_TIME.summary[groupname]['stddev']:.4f}"
    fps = f"{n_frames / _TIME.summary[groupname]['mean']}"
    print(f"{groupname}: time: = {mean} +/- "
          f"{stddev}"
          f" or FPS = {fps}")
    return {"groupname": groupname, "mean": mean, "stddev": stddev, "fps": fps}

Not sure if it is a bug or expected behaviour, but it seems that a requirement is that all summary metrics are calculated after all timings are complete, is that the case, if so might best to add to the docs.

Also is there a way to reset the cache, so we can start from fresh timings using the same names? Tried
from timing.cache import TimingCache
TimingCache.clear()

But seems to cause key errors doing so

Also is there a way to reset the cache, so we can start from fresh timings using the same names? Tried
from timing.cache import TimingCache
TimingCache.clear()

But seems to cause key errors doing so

@bml1g12  Sorry for super-late response here :sob:
I think the counter-intuitive part comes from the "caching" nature of _TIME.summary -> the implementation of .summary is something like:
        if self._summary is None:
            self.summarize()  # this re-generates self._summary
        return self._summary

So if you refer to the .summary and then run additional benchmarks, your summary will be out of date. This is intended due to the how big the overhead can get when generating statistics on very big collection of timings. So first reference to .summary will run summarize() (i.e. the costly calculating function) but subsequent will not.
I'd highly recommend either keeping the 1st approach (run and then analyse stats), or if you need to calculate intermediate stats, directly call _TIME.summarize() to do that as necessary.
If you don't care about penalty in your get_timings function, you can even just run _TIME.summarize() on the 1st line of the function. I don't think this should be too costly if you have relatively few timings to generate stats for.
I've added a a following docs note on the .summary to be released soon to PyPI (in upcoming version 0.5.1).
    @property
    def summary(self) -> t.Dict[str, t.Any]:
        """Return a collection of statistics for the timings in this group.

        Calculate statistics if not already calculated, and use cached values otherwise.
        """
        ...


Feature Request: Add Images to Existing Task

Actions before raising this issue

[X] I searched the existing issues and did not find anything similar.
[X] I read/searched the docs

Is your feature request related to a problem? Please describe.
Currently, it seems there's no straightforward way to add new images to an existing task in CVAT. This limitation forces users to create new tasks or use workarounds when they need to expand their dataset, which can be time-consuming and disruptive to the annotation workflow
Describe the solution you'd like
Ability to add new images to an existing task directly from the task details page
An "Upload Images" or "Add Images" option in the task's "Actions" menu
Maintain all existing task settings and annotations when adding new images
Describe alternatives you've considered
Creating a new task: This works but results in fragmented datasets and requires replicating task settings.
Additional context
No response

Hello
Thanks for the feature request, but we don't plan to support this at this time.
Project is an existing concept that already solves this problem:

You can set project-level labels for all tasks in the project
You can add images to the project by creating a new task


donejs-cli is not working correctly

@daffl let me know if you have something found out.

@pYr0x  this will get fixed by #4


suggestions for improvements

For safety, some company forbidden all the requests to unknown host, which results in error report to sentry.io receive 403 forbidden.
Hope sentry can give user privileges on how errors being handled.  Like tracekit,
tranceKit.report.subscribe(function report (errorReport) {
   // handle error by user config
});

I'm not sure what feature are you missing right now. We already provide things like dataCallback or transport that should allow you to do most of the things you'd ever need.

Yeah, dataCallback meet my needs.
Should read instruction seriously before open issue next time. [embarrassed]


Save user requests and restore them on reload grid

It would be nice that we could save the user's requests and then restore them on reload page.
This is because you can navigate (redirect) to another page (example edit page) and then return to the grid in the same position (status). It is possible to do it somehow? It is something that will be done in the future? Thank you

I wrote some JS to save the page size to session/ profile (on the server) and then restore it. I have not needed to do the same for selected columns, but that would be easy enough in the same way.
For stuff like column filters, those can be reconstructed from the server (for ASP.NET at least) and then when bootgrid fires Ajax requests it reads those values and... it works as you suggest.
Summary: I'm not sure that's really a Bootgrid problem, you need to do it yourself.

A nice option would be to add an option to save the state in the local storage. That would not require any server-side action but would help with scenarios like mentioned in the first post.


cluster-create now tags the OS Domain

Description

Updated cluster-create to tag the OS Domain with the cluster name
Minor fixes to the detailed design diagram

Tasks

https://github.com/arkime/aws-aio/issues/114

Testing

Add unit tests
Tested in my own AWS account by running cluster-create on an existing domain and making sure the tag appeared:

License
I confirm that this contribution is made under an Apache 2.0 license and that I have the authority necessary to make this contribution on behalf of its copyright owner.

LGTM, do you think in the future we should also do other high level items like S3 bucket and VPC?

Not a bad idea.  Maybe we should tag all the "sticky" resources that are likely to be left over after a default cluster-destroy, which includes the OS Domain, the VPC, the S3 Bucket, the KMS Key(s), etc...


Cannot read property 'length' of undefined

VSCode Version: Code 1.16.1 (27492b6bf3acb0775d82d2f87b25a93490673c6d, 2017-09-14T16:38:09.760Z)
OS Version: Windows_NT ia32 10.0.15063
Extensions:

Extension
Author (truncated)
Version

html-snippets
abu
0.1.0

vscode-angular2-files
ale
1.3.10

ng-template
Ang
0.1.7

typings-installer
ben
0.2.0

bracket-pair-colorizer
Coe
0.10.9

angular2-snippets
dan
1.0.16

tslint
eg2
1.0.7

angular2-switcher
inf
0.1.4

Angular2
joh
2.7.0

angular2-inline
nat
0.0.17

typescript-hero
rbb
1.4.2

autoimport
ste
1.2.2

Angular2
UVB
0.4.1

(2 theme extensions excluded)

Steps to Reproduce:

Reproduces without extensions: Yes/No

I'm pretty sure this is typescript hero. Got the same errors - they disappeared after uninstalling that extension.


Remove shapeless as a dependency for the published libraries

Shapeless was only being used for Lazy for the legacy coproduct support. We can move the dependency to the test configuration and remove the legacy coproduct support.

Locally the presentation compiler test failed for me. It failed on the original master branch as well as this new branch. I may have been running that particular test incorrectly.

I think it is been failing for a while.
The presentation compiler isn't enabled in CI. I forgot to create an issue about that.

Codecov Report

Merging #405 into master will increase coverage by 0.73%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #405      +/-   ##
==========================================
+ Coverage   82.74%   83.48%   +0.73%
==========================================
  Files          27       27
  Lines         226      224       -2
  Branches        3        3
==========================================
  Hits          187      187
+ Misses         39       37       -2


Upgrade cassandra driver 4.10.0

Motivations
Unit tests with the OSS Cassandra driver version 4.10.0
Modifications

Upgrade the Cassandra driver.

Tests
Exclude the fasterxml (core+databind) from the cassandra driver deps to avoid an issue with docker-java. All unit tests are ok.

Any chance to get this merged?

Is it planned to support Cassandra 4?

Cassandra 4.x is already supported, see this test. However, there is a limitation when is used with withInitScript(). Our recommendation is to use something like liquibase-cassandra

The issue is not related to Cassandra 4.x, but about the cassandra driver 4.10+

A new implementation has been added in #8616 and it doesn't rely on the cassandra client.


Create a maintainers guidelines

@bartaz @jdalton I have a few suggestions on how to approach merging PRs, opening as an experiment for discussion:

Require one Github review approval, otherwise, block merging (including admins).
Always use Github "squash and merge" instead of the other options, leveraging the idea of one concern per PR.
Any additional things we should make the standard?

Note: Maybe 1 doesn't matter much since we don't have a lot of active maintainers?

Bump.

I'm laid back, whatever process you all dig is fine by me.

I'm fine with that. As you said, 1) doesn't matter that much at this point.

I agree with @bartaz, at this point makes no sense requiring strong review, although it's always desirable :D.
See https://github.com/impress/impress.js/pull/592.

Nice to see the project is still alive and doing well :)

@and3rson It's slowly going, we are focusing on the tests now. Any help is appreciated.

I guess situation is much simpler now compared to when this ticket was opened, so I will just state the guideline here and close this ticket: Going forward, I will review all PRs.
I will use both rebase-and-ff and squashing when convenient, time will tell if it's always meaningful and practical.
Project will follow meritocracy, so when we hopefully get more contributors now, those that show a sustained interest in contributing, can then get responsibility to be reviewers in the areas they have contributed to and are intimately familiar with. (If we get that far, writing down some common guidelines, including about coding style / structure and so on, may be needed.)


Quieted logrotate script

This is a fix for https://github.com/StackStorm/st2/issues/3633
The following updates have been made:

Redirect stdout to /dev/null for every st2ctl reopen-log-files command
Added missingok to /var/log/st2/st2web.log because this file does not seem to be created (at least on my Red Hat test box)

I just tried booting up an st2vagrant instance and /var/log/st2/st2web.log is missing there also.
Is this a legacy log file that needs to be removed from the rotate script?

Hm, is this a desired behavior?
In case log rotation fails, I imagine service could print some useful info to stdout (in addition to stderr) which could help with troubleshooting things.

As far as st2web goes - Yeah i believe it's an old service since now st2web files are served directly by nginx.
Need to check default nginx config to see where st2web vhost stuff is logged.

Normally we direct any "standard" information to log files and then any errors get raised so an email is generated.
Working in an environment right now where we send all emails generated by root user on linux hosts to a sysadmin distribution list. Any cronjob that writes to stdout generates an email and spams the whole DL. In our environment we have about several StackStorm VMs, each sending us a nightly email with the content shown in the issue linked in the original post. This spam behavior is not desired, especially since there are no errors and this is simply rotating logs. We would want to be notified however if the rotation fails and we need to take some sort of action (example, maybe the service dies when logs are rotated).

@nmaludy Yeah, something like that could work :+1:

@armab @Kami Small update based on your comments.
I tried initially to go with @Kami 's approach of redirecting both stderr and stdout to a file, but also emit stderr when logrotate was invoked so that an error email was generated.  There ended up being more code in the postrotate block to handle log redirections than to restart the service. Also, this code was quite nasty and hard to follow.
I went with @armab 's approach because it was much simpler and straight forward to implement.
The final change i made was to consolidate the standard and audit logs for a few of the processes. This reduces duplicate code and also allows for a single restart of the service instead of one restart for the standard log rotation and another restart for the audit log rotation.

If @armab is fine with just redirecting it to /dev/null, I'm fine with it as well :)

As far as "consolidating it" goes - we intentionally had two sections because those are two very different log files.
For the audit ones, users might want to rotate it less often and have different log rotation rules apply to it.
Having said that, if the default values are the same, I'm fine with consolidating it as long as we add a comment and document how to "decouple it" (aka add new section and have different rules apply to audit log files).

@Kami i've split the configs for the various log files out again. Let me know if you have any other thoughts.

Thanks for working on this :+1:

Also, to clarify:

The final change i made was to consolidate the standard and audit logs for a few of the processes. This reduces duplicate code and also allows for a single restart of the service instead of one restart for the standard log rotation and another restart for the audit log rotation.

We don't actually restart a service - we send it SIGUSR1 signal which causes the service just to re-open the log files without a restart.

Looks like we reached consensus here.
Thanks everyone, merging :+1:


Unable to get author of message

So as a standard I acquired while coding discord bots, I would prefer if my bot didn't answer to other bots.
However this reveals to be quite difficult to achieve since it seems the cache never has the message author stored.
This is the piece of code I have to attempt to get the message author from the cache.
ctx.cache.user(&msg.author_id).await

In reality, this always seems to return None. I noticed a trait for the Cache, but I'm not quite sure if that is being used or not.
Since I'm a rust noob, I'm almost certainly missing something. Can you help me again?

If it is returning None it is because the user is not stored in the cache. They are only stored when the bot is started and a Ready event is received with the data. The API has been updated, I'm not sure if users just aren't sent in that event or something has changed, sorry for the lack of maintenance, maybe I'll get back to doing something this week.
In case you really don't want to respond to bots, you can fetch the user in the api and check.
if let Ok(user) = User::fetch(&cx, &msg.author_id).await {
    if user.is_bot() {
       return;
    }
}

I don't recommend doing this, because of latency and the new rate limits.

If it is returning None it is because the user is not stored in the cache.

That I pretty much had expected. Even thou I did saw the code that is supposed to update the cache and where that is being called, I am not entirely sure it is ever executed.

The API has been updated, I'm not sure if users just aren't sent in that event or something has changed, sorry for the lack of maintenance, maybe I'll get back to doing something this week.

Good luck with that. I could try to help if I have the time, but no guarantees. XD

I don't recommend doing this, because of latency and the new rate limits.

Ye, not really gonna bother much about it then, it's not a current top priority. I have a working version that doesn't bother with the users being or not bots.


Test Adapter does not find addins

We are trying to write an NUnit plugin for FsCheck (github.com/fsharp/fscheck). The addin works fine with the standalone test runner, although it is not shown in the addin dialog. According to this converstaion in the mailing list, this is because the addin is built with .NET 4.0:
https://groups.google.com/forum/#!topic/nunit-discuss/je0VXIsVQNQ
The addin is also written and compiled using F# (3.0).
The addin does not work with the test adapter - we've tried putting the addin and its dependencies in the addins folder where the adapter lives, as well in the folder itself. It looks like it is never loaded.
It's not clear how to go about writing an addin for NUnit that works with the standalone runner, the test adapter, the test adapter NuGet package, and Resharper.
From https://bugs.launchpad.net/nunit-vs-adapter/+bug/1193957

I agree this is 3.0 only. That said, the meaning changes substantially in 3.0. We will have to decide

whether to support extensions
whether to support user extensions
how users can install their extensions.
In the end, none of this has anything to do with NUnit V2 addins so a new issue should be written to refer to this one.

Issue moved to nunit/nunit3-vs-adapter #222 via ZenHub


Add an API isinstance(inst, cls) that mirrors the Python isinstance.

The current isinstance<T>(inst) tests if a Python object is an instance of a bound C++ class. This new overload tests if a Python object is an instance of a Python class. I have needed this helper in several nanobind-using codebases, and the same API exists in pybind11.

This looks great. Could I ask you to add a changelog entry?

This looks great. Could I ask you to add a changelog entry?

Done!

Thanks!


Teach status to recognize multiple files with identical contents

This pull-request teaches the git lfs status command how to recognize multiple files that share the same contents. By modifying the behavior of lfs.ScanIndex, we now maintain a slice of file metadata associated with a given OID, instead of a one-to-one association (as before).
ScanIndex now only accepts unique pointer OIDs (even if there are multiple pointers for a given OID, in the case of many files sharing the same contents), and then resolves them all at once to multiple files, per the data stored in the indexFileMap.
While working on this, @technoweenie and I found what may be a Git core bug having to do with checking a file that is both in the index and working tree (i.e., it appears in both git diff-index and git diff-index --cached). I stashed a failing test that I wrote to demonstrate this behavior in this Gist.
As an aside, this PR is a temporary measure. Maintaining these extra caches is not a long-term solution that I'd like to keep, this is more of a hack to solve this bug quickly.
--
/cc @technoweenie @sinbad @rubyist

Thanks for the review! 👋


Update 3rd party dependencies

pls check with the deps in ./es/build.gradle those are taken from ES. you should not force other versions.

@dobe Thanks! I've reverted some of the changes.

@dobe please check again


[usecase-embedding] Add a usecase for checking overScrollBy method

Impacted tests(approved): new 2, update 0, delete 0
Unit test platform: [Android]
Unit test result summary: pass 0, fail 2, block 0
Bug is reported
https://crosswalk-project.org/jira/browse/XWALK-4894

LGTM


GitHubRepo._clone_repo() should fetch updates to cached local repos if local repo already exists

I noticed today after I pushed changes to the test_staging branch in the main agentos-project/agentos repo that those changes weren't getting pulled down by AgentOS automatically.
I think I found the problem in GitHubRepo._clone_repo():
https://github.com/agentos-project/agentos/blob/54928a779e673c52e816c2bf84b675a597e45640/agentos/repo.py#L319-L330
If a local repo exists already, this function should call porcelain.fetch() on so that it gets updates form the remote.

I thought I had fixed this by adding a porcelain.fetch()  as in:
https://github.com/agentos-project/agentos/blob/9316319a6e1ccb03b3c349f20459b8fd39dc88a1/agentos/repo.py#L339-L347
But then when I ran that code in a debugger, I saw that command did run as:
porcelain.fetch(
    repo='/Users/andyk/.agentos/cache/repos_cache/agentos-project/agentos.git/test_staging',
    remote_location='https://github.com/agentos-project/agentos.git')
)

...and it didn't throw an exception but also didn't actually perform a git fetch in the repo at /Users/andyk/.agentos/cache/repos_cache/agentos-project/agentos.git/test_staging.
I determined this by running:
$ cd /Users/andyk/.agentos/cache/repos_cache/agentos-project/agentos.git/test_staging
$ git fetch
From https://github.com/agentos-project/agentos
   430344b..54928a7  master       -> origin/master
 + 2f830d5...9316319 test_staging -> origin/test_staging  (forced update)
   f191051..1a8803f  website      -> origin/website
 * [new tag]         v0.1.0       -> v0.1.0

You can see from the output that the remote branches were not actually fetched when my porcelain.fetch() command ran.
So I guess I'm probably using dulwich/porcelain fetch incorrectly.
@nickjalbert any insights?

This was closed automatically when I merged your PR, but I'm not sure it should have been (or, at least, we should confirm the bug is fixed).
I also ran into problems similar to this when swapping git for dulwich.  I'm still not sure if I misunderstood the porcelain API or the git internals, but I did spend a chunk of time looking at the dulwich source to trying to figure it out.  Finally, I timeboxed it and decided to return to it in the future.
Long story short, I found dulwich a bit hard to wrangle and I don't feel I have a great grasp on it.
About your specific question, I would have expected the fetch to work (note: I think then you'd need to update HEAD to the latest fetched version of your branch to get the behavior you wanted) so, alas, not much help there. 😭

Yeah the code also does the reset --hard after the fetch.
I was in a very similar situation yesterday to what you described where I ended up time boxing my digging into dulwich code. Worst case, for now we can delete local repos when we suspect the issue is with caches.
We should just keep in mind that other people might run into bugs like this.

This is definitely still a bug. I just run into this bug again when I tried running tests after merging your PR which switched the testing branch back to test_prod. I got an error as though my tests were running against an old version of that branch, so I deleted my cached agentos-project repo and re-ran my tests and they passed.

It's probably not a bad idea for the tests to clear all caches (or to not use any caches).  Potentially a separate issue from this...

Good suggestion @nickjalbert, just created #329 to track it

Clearing these caches can be tricky if modules that live in them have already been loaded into a python process. Just ran into this when trying to update the tests to clear the repo cache, but then Windows (though not Linux or MacOs) threw an exception during test_component_from_github_no_venv when trying to clear the cache because test_component_from_github_with_venv had already run and loaded some modules from that same local github repo.
I now think perhaps we should only cache repos locally into directories that have the name of the github hash, and then we can also separately cache the mapping between branch names and repos.

To record our discussion from yesterday:

cache repos locally into directories that have the name of the github hash

I dig this!
Another thing we could do is to set the cache directory to a tmpdir in the tests so there will be no reuse between tests.

https://github.com/agentos-project/agentos/pull/379 should fix this.

Fixed in #379


Create a test tracking spreadsheet for direct archiving

We need to track our testing regarding to direct archiving.
We need to come up with an agreed form to report the testing results.

Tracking sheet for testing


Custom claims are not always available

It seems fastapi-jwt-auth (current git HEAD) adds custom claims identity and type to the tokens it generates and also expects any incoming tokens to contain those claims. This is not the case if tokens are generated by a third-party provider. Is most cases identity can be replaced via standard claim sub, while type is mostly available via typ (which might not be standard).
Could you share the reasoning behind these claims?

first, I use identity claims because that package I use to make jwt its doesn't support to add sub claims which is refer to pyjwt, and the type claims its identify incoming token its access token or refresh token in some cases you only want to accept refresh token to your endpoint.

after I did research about claims information, I decide to change identity claim to standard claim using 'sub', thanks for your information 🙏


Move to React + Typescript + Bulma

[x] React
[x] Typescript
[x] Bulma

Pages

[x] Home
[x] Experience
[ ] Contact
[ ] Hobbies
[x] Certifications
[ ] Donate
[x] Communities

Cancel. I want to find another inspiration.


Create mockAPi and add index.tsx able to launch - createDevApp

Add index.tsx able to launch locally a backstage app within the plugin project
Mock an entity and KubernetesApi
[x] Still getting the issue about: No implementation available for apiRef{plugin.kubernetes-auth-providers.service} -> fixed with commit "c84f991"

Call Stack
 useApi
  node_modules/@backstage/core-plugin-api/dist/esm/StorageApi-c8a5e616.esm.js:54:11
 useKubernetesObjects
  node_modules/@backstage/plugin-kubernetes-react/dist/index.esm.js:400:99
 useK8sObjectsResponse
  src/services/useK8sObjectsResponse.ts:22:133
 QuarkusApplicationInfo
  src/components/QuarkusApplicationInfo.tsx:39:123

Still need to pass data that the plugin can display and which is matching the component

When correct data are mocked, then the kubernetes accordion is working.

So, there is certainly an issue with the context we provide to the Quarkus ApplicationInfo card which is perhaps related to this error:

I found the issue ;-)
The following code const componentName = currentPageLocation.pathname.split("/")[4]; part of QuarkusApplicationInfo.tsx file which is working when a component is registered as Entity part of a backstage catalog don't work at all when we create a devApp and set the path to access the Quarkus Console as such /quarkus as the component name is not included !
Until now, I aligned the path of the Page to register the QuarkusApplicationInfo as such "/a/b/c/my-quarkus-app" but we should certainly review how the componentName to be fetched from the watchResourcesData is calculated to avoid the issue.
createDevApp()
    .addPage({
        title: "Quarkus application info",
        path: "/a/b/c/my-quarkus-app",
        element: (
            <TestApiProvider
                apis={[
                    [kubernetesApiRef, new MockKubernetesClient(mockKubernetesQuarkusApplicationResponse)],
                ]}
            >
                <EntityProvider entity={mockEntity}>
                    <QuarkusConsolePage />
                </EntityProvider>
            </TestApiProvider>
        )
    })

@cmoulliard: Approved, but you might want to squash some of those commits before mergin.


MsBox.Avalonia.Enums.Icon.Info should be same as windows standard message box info icon.

Or It should allow custom Icon image file. I want to use something standard info ⓘ icon for message. But does not exists. Insteed it shows some search icon

it's allow custom image file

How to use costom image file? I did not find in documentation. And It also does not show modally in wasm platform.

How to use costom image file? I did not find in documentation. And It also does not show modally in web platform.

messageboxcustomparams imageicon


Add ja locale

I don't speak any other language so I added only ja locale :)

Great, thanks!

@nickcharlton I added i18n spec. Please let me know if I did something wrong :)
And the hound is barking... Should I add hound.yml or .rubocop.yml as well?

Great, thanks!
Hound is configured to use my "organisation config", which is the same as the thoughtbot one. Would you mind fixing up the comments it mentions?

@nickcharlton Sorry for the delay, now all green!


Add creating collective variables docs

Adds another "topic" to the docs, about creating collective variables. This includes several examples of increasing complexity.
This is a docs-only PR, so it should be easy to merge.

This should be ready for immediate review. Will merge it tomorrow (Thursday) if no comments by then. Purely additions to documentation.


(dev) Event loop - node events

Coverage increased (+1.6%) to 77.353% when pulling aa75acb13d9360a63ab2dee3b89f936fa6d649bc on milanlenco:dev into 433f88d602de1b80dde84d97d8c8d5ecb8fda0ad on contiv:dev.


Strange white mask in diff editor

monaco-editor version: 0.17.1
Browser: Chromium
OS: Windows 10
Steps or JS usage snippet reproducing the issue:
Recently I developed a tool for developers named SmartCommit based on Electron and Vue (https://github.com/Symbolk/SmartCommit ), which takes over the job of git-status, git-diff, git-add, git-commit and git-push commands. I provide two kinds of diff views with monaco diff editor in the two tabs of a modal r: side by side and inline diff.
However, I found a strange bug with it: the default diff view is ok (Figure 1), but when switching to the other diff view, there will be a strange white mask with a close button  (Figure 2) at the top-right corner, which must be clicked to see the diff view  (Figure 3).
Figure 1

Figure 2

Figure 3

Related code are open sourced at https://github.com/Symbolk/SmartCommit, which is easy to compile and inspect. Thanks for any help!

Hi @Symbolk, did you ever figure this out? I am getting the same thing.

Hi @Symbolk, did you ever figure this out? I am getting the same thing.

Unfortunately not.

I just figured it out on my side. By adding a layout() on the editor, I was able to get it to render properly. Hopefully, that will get you working as well.

@mattduffield Seems like it does not work for me. I put two diff editors in two tabs of a Vue componment called SweetModal (https://sweet-modal-vue.adepto.as/). When the modal is open, the editor in the default tab is always ok, but the editor in the other tab is always covered by the white mask.

Hmm, I also have a regular editor and then I allow the user to launch a diff editor. By calling layout(), it now works fine. Previously, it was covered like you are describing. I hope you figure it out.

Do you have a standalone reproducible case? That looks like the diff review which is available when pressing F7 (it is an accessible friendly way to navigate a diff).

This issue has been closed automatically because it needs more information and has not had recent activity. See also our issue reporting guidelines.
Happy Coding!


Missing extrastyle.css in the /css folder

Line 130 of motion.php is

however there is no 'extra style.css there. There is a extrastyle.txt that contains 'es_Default.css'. Becasue of the missing file, an error is showwing in the error.log:
[Wed Jul 22 10:46:48 2015] [error] [client <IP_ADDRESS>] File does not exist: /var/www/picam/css/extrastyle.css, referer: http://<IP_ADDRESS>/picam/motion.php

Also an issue on
line 311 of preview.php and
line 312 of schedule.php

The extrastyle,css is for user overrides of styling. User css can be put in there and then selected from main page. This is how the night mode is done.
We changed how that was selected on index page but not on schedule and preview.  I'll fix that.
Thanks


Will the plugin be updated for current ionic versions?

Hi @ihadeed,
as you probably know, there are some major issues with the super-tabs plugin and the current release of Ionic CLI 3.13.x, especially with run *platform* --prod.
For me, this plugin is one of the most important plugins, since it dominates the navigation process of the user and hence is visible almost all the time. However, with the current issues, it's not reliable anymore, as it used to be. There are issues with the css-file not beeing copied when building with --prod. And sometimes the tabs even fully disappear. Only solution for the user: Close and restart the app. No good UX...
Hence my question: Are you planning on updating this great plugin? Otherwise we had to roll back to an older version of Ionic which doesn't seem to be a great idea either...
Thanks for your answer. As said in other issue-topics, I really appreciate the work you are doing here. Thank you!

Hey @JaneDawson
Yes the plugin will be updated soon. thank you for your patience.
I think the --prod issues are from the ionic-app-scripts package and they should be fixed in the latest release (or nightly).


resize of remote image doesn't seem to work

I am not sure if this is a bug, or missing feature.  I was trying to cropResize a remote image, e.g.
![](https://www.nasa.gov/sites/default/files/thumbnails/image/juice_artist-impression.jpg?cropResize=300,300)

but the cropResize had NO effect on the remote image size.  Conversely, the following manual entry in the same grav page DOES work
<img src="https://www.nasa.gov/sites/default/files/thumbnails/image/juice_artist-impression.jpg" width="300" />

I guess there is a smarter way to resize the image with the <img /> call, but really, I was hoping there was a nice way to use the grav image linking ![]() method to resize.  Is this possible?  If not, is it easy to add?  Seems like there is some well known PHP ways to do this, but I am inexperienced enough with PHP and GRAV to know where I would make those changes.

Cropping only works for local 'page' media, not remote images.  Your width="300" example is not cropping its just setting the size of the IMG element to width of 300px. This is not a bug, it's a limitation/feature.


Improved Speed for Put File with many files

The overall throughput using pachctl put file is quite slow. On a co-located node on GCP, a 156GB upload of 1.2m files, was estimated to take ~33 days (2 days elapsed) to upload using the following command:
pachctl put file -r -p 252 --compress data@master -f ./
Created a node colocated - n2standard4 4CPU x 16GB ram

Download the full dataset (tar): Downloaded 137.7 Gigabyte in 1:44:55 hour(s). (22942.44 KB/s)
Extracted size: ~156GB
Formatted the data according to desired file structure for Pachyderm data repository and subsequent pipelines
Waited 2 days during an upload, and an estimated 31 days were left to finish. Measured based on network data in Prometheus metrics in Hub.

Recommendations for pipelines to be used to ingest the data are insufficient. They introduce an anti-pattern when working with datasets, serving data from other sources to ingest into Pachyderm at a future stage.
Uploading compressed versions of the data is possible, but changes/future commits to the dataset require storing duplicates of the dataset with minor changes.
@jrockway and @chainlink feel free to add more details.
What is the goal / desired outcome?
There needs to be a reliable way to upload data that is intuitive (like pachctl put) that can scale to a reasonably sized dataset. If pachctl put is only able to work with small amounts of data, we need to document this.
If there is a way to accomplish this today via workaround, what does that require?
There is currently no workaround today. Even with direct support, we have been unable to find a sufficient solution. Recommendations for using pipelines sacrifice the ability to iterate on the data in a Pachydermic way, and co-locating data to improve the upload speed, up until now, has been unsuccessful.
(Optional) What is your proposal for a feature to solve this?

Environment?:

Kubernetes version (use kubectl version):
Pachyderm Hub
Pachyderm CLI version (use pachctl version):
Pachyderm Hub

$ pachctl version
COMPONENT           VERSION
pachctl             1.12.5
pachd               1.11.9

Cloud provider (e.g. aws, azure, gke) or local deployment (e.g. minikube vs dockerized k8s):
OS (e.g. from /etc/os-release):

NAME="Ubuntu"
VERSION="16.04.7 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.7 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

Others:

We have made several improvements to make this faster. In addition we discovered during testing that for many small files the progress bars actually add significant overhead so you will get faster performance if you pass --progress false to put file. We'll continue to make progress on the performance of put file`.

Agent Dale Georg linked Freshdesk ticket 278 for this issue.


Sample configuration file

Just a suggestion.
It may be a good idea to package a sample toml file in the binary release to show all the options available. I wanted to run https and found that the [ssl] section in the documentation refers to the database connection to the database and not to the http connector itself.
Looking at the configuration code in config.go I found the (currently undocumented) [https] section with associated config.

Great idea... would be nice to receive a PR with this config file example!


chore: release main

:robot: I have created a release beep boop
base: 0.0.35
0.0.35 (2024-10-23)
Features

✨ make first base release (#175) (a7eb8c9)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

base-v0.0.35
:sunflower:


Fix a couple typos and do some rewording

Literally just a couple wording/typo changes because why not :p

Sorry, didn't see this PR.
There have been changes which also fixes the ones you wanted to fix here, so I'll close it.


Fix Issue 17228 - File size() can't determine file unsearchable on Wi…

…ndows

Rebased to hopefully get rid of the unrelated failure of std.socket on the auto-tester

The difference is that a 32-bit host Linux doesn't pass, but 64-bit Linux does (whether compiling for 32- or 64-bit)

That's really bizarre.

If you do testprogram < input, this may well succeed.

Yes, it does (on Linux).

It would probably be difficult to set up a true non-seekable file portably and reliably.

Maybe std.process.pipe()?
I don't have access to a WIndows machine right now, so won't be able to look at this closely for a while. Fixes welcome, edits are enabled.

Looking again with more history, the failures seem very intermittent. Something is definitely weird about this, almost as if garbage is being returned from tell.

Changed the unittest to use std.process.pipe, let's see how the autotester likes that.

Rebased. Maybe it has magically fixed itself by now.

Unfortunately, with time's passing, I am less and less friends with Windows. It would be nice if someone could take over this.

Anyone that wants to pick this up can find the link in the associated bug report. Until then, we can close this.


Allow constructing an Instructions iterator from a slice

Currently, the only way to construct an Instructions iterator is from a Script, which contains a Box<[u8]> and thus requires an allocation. This precludes iterating over the instructions in a parsed Witness struct without an additional allocation.
This PR adds Instructions::new and Instructions::new_minimal, which allows iterating over a slice directly, not enforcing or enforcing minimal pushes respectively.

I think this is more cleanly solved by #1155 which I hope will go into the next version.

Just changed the names to from_raw_bytes and from_raw_bytes_minimal. #1155 definitely seems nicer, I suppose it just depends on how long before it's merged.

@casey it should go to the next version.

Closing because we merged #1155, please holla if I'm missing something @casey.

I'm super happy to use #1155 instead. Do you know when it'll make it into a release?

We should be gearing up for a release in about 2 months - we once said we would like to release every six months approximately. Does that work for you?

Honestly, this release is crazy so I wouldn't be surprised if it took longer. :(

That is what I'm concerned about, crate smashing is going to take for ever, I think we need to draw an arbitrary line in the sand and feature freeze otherwise we will not get a release out for too long. My suggestion would be as soon as hashes has the hex module removed - concept ACK/NACK?
(This is quite an arbitrary place to be discussing this, so if there is discussion here I'll cross post a note in discussions some place).
tag: @apoelstra @sanket1729 @Kixunil

So the release would be just reorganizing modules without splitting into separate crates yet. ACK this idea. There are many other things I originally wanted to do but probably best to not prolong the release.

Moving this to discussions: https://github.com/rust-bitcoin/rust-bitcoin/discussions/1501

We should be gearing up for a release in about 2 months - we once said we would like to release every six months approximately. Does that work for you?

Is there a reason to release so infrequently? Six months seems like a long time between releases. I would aim to release at least a couple of months, which might prevent feeling the need to rush things into a release or delay a release.

Yeah, I'd like to release more often. But I think it makes sense to keep it at 6 months while doing large refactorings like we're doing now.

Is there any way to make those large refactorings more incremental?

In principle we could cut them at an arbitrary point (all git commits are tested) but if the goal is to e.g. flatten the util module then releasing in the middle of that work would feel strange.

I agree. I also don't like the idea of doing major releases super frequently, especially while we're doing API changes that are likely to change multiple times in circles. It would be horrible if I had to deal with a dozen releases every time I revisit infrequently-maintained projects.
On a good week we've got 3 people maintaining this project, only 2 of which are ever online at once. On a bad week that number is more like one.
I'm happy to revisit this once the crate smashing is (mostly) done. But then I'd also like to get into a cadence where we treat breaking changes as a serious/infrequent thing.

That makes sense! Could you use public re-exports to do crate-smashing in a backwards compatible way? E.g. to move X to another crate, you make that crate a dependency of rust-bitcoin, move X to that crate, and then re-export X from rust-bitcoin? Eventually, rust-bitcoin has the same structure, but with a bunch of re-exports from other crates. Then, you can do a single backwards incompatible release where you change the structure of rust-bitcoin to be how you want it, but the actual PR is relatively small, since it's just moving things around.

I definitely feel the pain of having limited developer bandwidth. Is this something that grants could help with? I know that sometimes, the bottleneck isn't money, but sometimes it is, e.g. if there's a motivated contributor that would like to take time off to work exclusively on the project, but can't due to financial constraints.

Grants don't seem to be super easy to come by right at the moment :)

Maybe it's a good fit for https://opensats.org/?

Could you use public re-exports to do crate-smashing in a backwards compatible way?

What you describe is actually exactly what we plan to do. But we would also like to stabilize individual parts starting from the simplest so if you happen to use those you'd get more frequent feature updates and infrequent (never?) breaking changes.
We couldn't do it in opposite order because there were (and still are) tightly-interconnected things.

Is this something that grants could help with?

It feels like the bigger issue is timezone combined with the (very reasonable!) requirement of two ACKs. Some things did improve but some got worse. Funding may might marginally help, hard to say.

Maybe the ideal grant would be for a contributor that only reviewed things 🤔

Interesting idea! Faster response time to misakes would be nice. But still it happens that it takes multiple of us to notice problems, I sometimes only notice when reviewing PR ~the third time.


Deprecate ProgressBar and rename it to TQDMProgressBar

What does this PR do?

Deprecate ProgressBar and rename it to TQDMProgressBar.
Follow up PR would be updating the tests and docs.
Does your PR introduce any breaking changes? If yes, please list them.

Before submitting

[ ] Was this discussed/approved via a GitHub issue? (not for typos and docs)
[x] Did you read the contributor guideline, Pull Request section?
[x] Did you make sure your PR does only one thing, instead of bundling different changes together?
[ ] Did you make sure to update the documentation with your changes? (if necessary)
[x] Did you write any new necessary tests? (not for typos and docs)
[x] Did you verify new and existing tests pass locally with your changes?
[ ] Did you list all the breaking changes introduced by this pull request?
[x] Did you update the CHANGELOG? (not for typos, docs, test updates, or internal minor changes/refactorings)

PR review
Anyone in the community is welcome to review the PR.
Before you start reviewing make sure you have read Review guidelines. In short, see the following bullet-list:

[x] Is this pull request ready for review? (if not, please submit in draft mode)
[x] Check that all items from Before submitting are resolved
[x] Make sure the title is self-explanatory and the description concisely explains the PR
[x] Add labels and milestones (and optionally projects) to the PR so it can be classified

Did you have fun?
Make sure you had fun coding 🙃

@awaelchli @Borda Would need any of your approval for this.


Workflow fail silently

After the changes in #130, the GitHub action will fail to detect errors during the execution of any command since the shell will catch the error and prevent it from propagating to the action. The GitHub action will require updates to avoid situations like this happening.

Maybe we can remove the ./build_test_design.sh at the smoke test and force to use the synthesis result and the place and route result from the CLI.


Style model pages added

Related Ticket(s)
https://www.ibm.com/standards/web/ibm-dotcom-library/guidelines
Description
Adding the MVP version of the style model pages.
Changelog
New

./src/pages/guidelines/ibm-dot-com-style-models/**/*

Changed

./src/data/nav-items.yaml

@wonilsuhibm what do you think about removing IBM.com from the left nav and page lead space for style models? Looking at the page, IBM.com is repeated many times but I wonder if that's necessary?

When an adopter is on the IBM.com Library website, isn't the expectation that all of the content and guidance on the site is specifically tailored to IBM.com? We don't label our components or patterns as "IBM.com components" or "IBM.com patterns" in the navigation so I wonder if we could remove it from style models for consistency and simplicity?
I think it still makes sense to have IBM.com style models written out in full in the description/intro/main body text.

Hey this is great to see! I wanna leave a few general comment just for consideration:

I noticed a lot of "floating images", showing up right after a H4 (16px semibold) and no image caption. In Carbon sometimes this happens in components documentation because it's pulled from github's README. We never liked it because I think H4 should technically be followed by a body copy, or the images/media should get a bit more context via captions.

Inconsistent image background. Some masthead examples are contained in a background, sometimes not. I think this could an opportunity because unique to Dotcom library, everything is relative to the browser. So maybe there should be a standard schema for "browser" repeatedly used in the image examples to frame how components are patterns are expected to be used in situ, and maybe always use a 75% zoom scale etc:

Since this is a guidance site, everything should be exemplary. Here's a moment where link CTA that can be more action oriented:

"The design specs and functional specs for Link with icon can be viewed here."

Maybe change to

"For more details, view design and function specifications."

I will leave comments by section:
Overview > Our vision

A very subjective opinion -- I feel like this section should be called something else. "Rationale" or "Principles" maybe? "Our vision" sounds a bit broad and should live under "Getting started > Overview" or a site-wide "About" page
Starting and ending touch point's dots were missing in the illustration?

Overview > What are

seems to be the most useful section. Personally I'd suggest trimming down the vision, so this section is closer to the top.
Should "Primary user intent" in the paragraph be bolded or italicized? I see the term reused later so it might want a bit more emphasis here.

I think the images miss caption or text inside to help clarify what I'm looking at. Maybe annotate with "3 - 1 empty - 10" etc

Overview > Using style models

"Using style models" is repeated on both tabs. I'd suggest remove from "Overview" and consolidate on the second tab.

Style model list


RCSBuildAid version 0.7

I'm not familiar with the spec, I just copied and edited the relevant stuff.

Since you use github and avc you could create a netkan file which would automatically generate the ckan files with each release. However that would use the github release api which does not seem to work with your repo. https://api.github.com/repos/m4v/RCSBuildAid/releases

I need to do a release in github for it to show in its API, I just use it as a repository. As I understand, for that netkan thingy to work I will have to manually do a release in github with a  zipped binary and all that.
Either way I need to do something for get ckan to pick it up.

Please filter the Sourcecode out, we don't normally install that.

how do I do that?

in the install stanza add a "filter" : "Sources" so it becomes
"install": [
       { "file" : "RCSBuildAid", "install_to" : "GameData", "filter" : "Sources" }
    ]

That should filter the entire Sources folder.

done

Thank you :D


Runtime doesn't reload module's sources after LS reloaded the buffer

This task is automatically imported from the old Task Issue Board and it was originally created by Hubert Plociniczak.
Original issue is here.

This is causing unexpected failures in positions e.g.,

org.enso.interpreter.service.error.FailedToApplyEditsException: Some(Failed to apply edits for local.Unnamed_35.Main, edits=***, failure=InvalidPosition(Position(13,369)), source='***')

or
org.enso.syntax.text.Parser.ParserError: Some(Internal error in parser Could not not deserialize metadata (found 2 metadata sections))

Comments:
Hubert Plociniczak reports a new STANDUP for yesterday (2023-01-25):
Progress: Investigation revealed that we modules sources weren't being reloaded from FS in runtime. That is why all buffers in LS were OK and why applying edits was crushing only in runtime. Draft PR ready and seems to do the job. It should be finished by 2023-01-26.
Next Day: Next day I will be working on the #184324365 task. Figure out the proper test for the bug. Address PR reviews. (Enso Bot - Jan 26, 2023)

**Hubert Plociniczak** reports a new **STANDUP** for today (2023-01-26):
Progress: Finished PR along with a decent test for it. Also spent a while trying to address reviews in various PRs. It should be finished by 2023-01-26.
Next Day: Next day I will be working on the #184208157 task. Looks like there are some issues with initializing git on windows. (Enso Bot - Jan 26, 2023)

Closed by #4083.


Fixed-position stat col headers

The column headers for term statistics should be fixed so that when a user scrolls down the headers are still visible.
Also, we need a way to limit which of these statistics are shown (checkboxes, maybe?).

Fixed: done
Limit statistics: done
However, like Vaccari said (and I also have this issue), the first row in candidate terms hides behind the header a bit.


Temporary limit increase has OLD date?

Temporary increase says through June 30th... Surely this is now for the rest of 2021?
Just like the 1K meetings and overflow was extended for all customers to entire 2021 (from June 30th)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 905e2c09-ef3e-07cc-0809-6ff171157c47
Version Independent ID: c0d33162-183d-fff8-32e9-a999f391ea51
Content: Limits and specifications for Microsoft Teams - Microsoft Teams
Content Source: Teams/limits-specifications-teams.md
Service: msteams
GitHub Login: @SerdarSoysal
Microsoft Alias: serdars

Thank you for submitting feedback.
I think the best way forward is if you open a service ticket in your tenant so this can get resolved ASAP. Based on the outcome let me know if it is something that can be called out in the docs.
Please follow this link to contact support for business products:https://docs.microsoft.com/office365/admin/contact-support-for-business-products
Please keep us posted here on the resolution so we can feed whatever information you discover into the content.


Allow for flexible boolean mask similar to numpy

Having flexible boolean masks would be something of advantage for the whole community. At the moment of writing using TF version 1.12.0 in order to construct a boolean mask one has to predefine the mask and use it using a specific function tf.boolean_mask. Instead it would be much more productive to have similar functionality that is found in numpy. For instance:
x = tf.random.normal(10, 4)
x[x > 0.01]

similar to
x = np.random.randn(10, 4)
x[x > 0.01]

System information

TensorFlow version (you are using):
1.12.0

Describe the feature and the current behavior/state.
At the moment there's not an equivalent way of easily creating and applying on the fly boolean masks, except maybe from the tf.where
Will this change the current api? How?
Yes, it allow for more flexible and productive api.
Who will benefit with this feature?
Community

To create a boolean mask you can do tf.where(boolean_tensor) and to apply it you can do tensor[tf.where(boolean_tensor)].

Actually my example was wrong, but I don't see what's the issue with tf.boolean_mask(a, a > 5) or something like that.
I'm happy to consider better syntax, if someone is willing to contribute the code.


Update Play nginx config to run best-effort queries.

This change is

Nginx on Play has already been restarted with this config.