AmazonS3Client.CopyObject() is incorrectly reporting missing DestinationKey

Description
When copying an object using Amazon.S3.AmazonS3Client.CopyObject, if the DestinationKey is set to "/", then an exception is thrown reporting that the value is not set.
Use Case
I have an actual object "/" (without the quotes) in my bucket that I want copied.
Reproduction Steps
var client = new Amazon.S3.AmazonS3Client(Amazon.RegionEndpoint.USEast1);

var request = new Amazon.S3.Model.CopyObjectRequest()
{
  SourceKey = "/",
  DestinationKey = "/",

  SourceBucket = "bucket1",
  DestinationBucket = "bucket2",
};

try
{
  var response = client.CopyObject(request);
}
catch (Exception e)
{
  Console.WriteLine("{0}", e.Message);
}

Console.WriteLine("Press Enter to continue...");
Console.ReadLine();

The above code throws an exception and outputs the following:
DestinationKey is a required property and must be set before making this call.
Parameter name: CopyObjectRequest.DestinationKey
Press Enter to continue...

Environment

SDK Version: AWSSDK.S3 version <IP_ADDRESS>
Package Version: AWSSDK.S3 version <IP_ADDRESS>
OS Info: Windows 10
Build Environment Visual Studio 2019
Targeted .NET Platform:  4.7.2

Additional Notes

This appears to only happen when the object is "/". Copying of other objects works correctly.

This issue occurs without a valid AWS role or key. So it appears to be an SDK validation check before the s3:CopyObject API is actually called.

This is a :bug: bug-report

Hi @mwhouser,
Good morning.
I don't think this is a bug. The Source and Destination keys must point to valid object. The character / specifies the folder structure, not the actual object. Kindly note that this is not similar to existing bug https://github.com/aws/aws-sdk-net/issues/1241 where CopyObject fails for file starting with "/" .
If you want to copy the folder structure then you might have to write logic that iterates through source folder objects and copies them one by one. The CopyObject API call handles single object.
Hope this answers your question.
Thanks,
Ashish

Hi @ashishdhingra

The Source and Destination keys must point to valid object.

It does. "/" is a valid object key.

The character / specifies the folder structure, not the actual object.

That is not correct. "Folder structure" is a human concept laid over S3 as an organizational system. S3 keys are just that... keys. The / characters in keys don't mean anything to S3 aside when used by the APIs as delimiters. And as such, I can choose to use any other character as my "Folder structure" delimiter... such as an M character if I so choose.
I can create objects with all 3 of the following keys and they will all be distinctly different objects:

"abc"
"/abc"
"abc/"

Try it yourself with the following sample program. Also notice that I am writing to an object called "/" and reading it's contents too. So I should be able to copy that object.
static void PutStringsDemo(Amazon.S3.IAmazonS3 client)
{
    string bucketName = "bucket1";

    PutObjectString(bucketName, "abc", "This is object 'abc'", client);
    PutObjectString(bucketName, "abc/", "This is object 'abc/'", client);
    PutObjectString(bucketName, "/abc", "This is object '/abc'", client);
    PutObjectString(bucketName, "/", "This is object '/'", client);

    Console.WriteLine("Object abc: {0}", GetObjectString(bucketName, "abc", client));
    Console.WriteLine("Object abc/: {0}", GetObjectString(bucketName, "abc/", client));
    Console.WriteLine("Object /abc: {0}", GetObjectString(bucketName, "/abc", client));
    Console.WriteLine("Object /: {0}", GetObjectString(bucketName, "/", client));
}

static void PutObjectString(string bucketName, string key, string value, Amazon.S3.IAmazonS3 client)
{
    using (var stream = new MemoryStream())
    {
        using (var writer = new StreamWriter(stream, Encoding.UTF8, 1024, true))
        {
            writer.Write("{0}", value);
            writer.Flush();
        }

        stream.Seek(0, SeekOrigin.Begin);

        var request = new Amazon.S3.Model.PutObjectRequest()
        {
            Key = key,
            BucketName = bucketName,

            InputStream = stream,
        };

        var response = client.PutObject(request);
    }
}

static string GetObjectString(string bucketName, string key, Amazon.S3.IAmazonS3 client)
{
    var request = new Amazon.S3.Model.GetObjectRequest()
    {
        Key = key,
        BucketName = bucketName,
    };

    using (var response = client.GetObject(request))
    {
        using (var stream = response.ResponseStream)
        {
            using (var reader = new StreamReader(stream))
            {
                return reader.ReadToEnd();
            }
        }
    }
}

When I execute this program, I get the following output:
Object abc: This is object 'abc'
Object abc/: This is object 'abc/'
Object /abc: This is object '/abc'
Object /: This is object '/'

After executing this program, I have 4 objects in my bucket, as seen by s3:ListObjects:
$ aws s3api list-objects --bucket bucket1
{
    "Contents": [
        {
            "LastModified": "2020-09-25T20:13:36.000Z",
            "ETag": "\"03f9b0ffc1d9c2286a608660775d35a4\"",
            "StorageClass": "STANDARD",
            "Key": "/",
            "Size": 21
        },
        {
            "LastModified": "2020-09-25T20:13:36.000Z",
            "ETag": "\"92f596e88248ff7f4b11d774636be935\"",
            "StorageClass": "STANDARD",
            "Key": "/abc",
            "Size": 24
        },
        {
            "LastModified": "2020-09-25T20:13:36.000Z",
            "ETag": "\"fad4fd3092dd59db8ea2fa52dbd49996\"",
            "StorageClass": "STANDARD",
            "Key": "abc",
            "Size": 23
        },
        {
            "LastModified": "2020-09-25T20:13:36.000Z",
            "ETag": "\"9917757841faaf1fff236375e59aa749\"",
            "StorageClass": "STANDARD",
            "Key": "abc/",
            "Size": 24
        }
    ]
}

Notice the line in the first object:
"Key": "/",

So there's now an actual object of 21 bytes called "/" in my bucket.
When I examine the bucket using the AWS Management Console, I see the following:

Notice a few things:

There's a folder with a "blank" name. That's the "/" object.

Navigating into that folder will show me the "/abc" object.

There's another folder called "abc". That's "abc/".
There's an object called "abc"  with the same name as the "abc" folder. That's "abc".

Unfortunately, the AWS Management Console does not allow me to download the contents of "folder" objects, even though it has data.
So, to S3, the "/" characters means nothing and has no additional meaning. It's just another character in an object key.
TL;DR
It's possible to Put and Get an object named "/". So there should be no reason why I cannot copy it to another bucket.

Hi @ashishdhingra
One more thing. I can copy the "/" object using the follow AWS CLI command:
$ aws s3api copy-object --bucket bucket2 --key '/' --copy-source bucket1//

Note the double slash in the --copy-source parameter.

@mwhouser Thanks for your inputs. I would need to run some tests against the service API directly to see if it allows copying folder (key '/') from source to destination. I'm not sure if AWS CLI has any custom implementation. I will keep you posted.

Hi @ashishdhingra
I found the problem.
Please see this sample code:
var request = new Amazon.S3.Model.CopyObjectRequest()
{
        SourceKey = "/abc",
        DestinationKey = "/abc",

        SourceBucket = "bucket1",
        DestinationBucket = "bucket2",
};

var response = client.CopyObject(request);

I executed this code then I checked my target bucket. I don't see "/abc". Instead I see an object "abc". It's contents are the contents of "abc" in the source bucket. So the wrong file was copied. This is very, very bad.
Looking through the source code here on GitHub, I found the following:
https://github.com/aws/aws-sdk-net/blob/641affd215ac31454f9d034b8867a4f5adae76ef/sdk/src/Services/S3/Custom/Model/Internal/MarshallTransformations/CopyObjectRequestMarshaller.cs#L38
It looks like the SDK is intentionally stripping the leading slash from the source and target keys. Again, very, very bad.
So the problem is that:

The SDK is stripping the leading slash, then
The target key is now blank, so it's throwing the exception.

This behaviour is very, very bad. The SDK should not "fix" my keys for me. If I want to copy "/abc", then it should copy "/abc".

Do not close.

Still need to investigate.

Apologies for the long delay, but we just released a new version of S3 that does not remove the leading slash from object keys: https://www.nuget.org/packages/AWSSDK.S3/<IP_ADDRESS>
Changelog: https://github.com/aws/aws-sdk-net/blob/main/changelogs/SDK.CHANGELOG.2023.md#376740-2023-10-31-1833-utc

Update: We had to revert that S3 change as it caused an unexpected issue in our AWS Tools for PowerShell Cmdlets.
I'll discuss with the team what best approach forward is, and I'll provide an update here when I have one.

Another update: Starting in version <IP_ADDRESS> of the AWSSDK.S3 package, there's a new option in CopyRequest and CopyPart: DisableTrimmingLeadingSlash; if it's set to true the .NET SDK will not remove leading slashes from the object keys:
// Destination object on S3 will be "/destkey" (and not "destkey" as before)
s3Client.CopyObject(new CopyObjectRequest
{
    SourceBucket = bucketName,
    SourceKey = "/sourceKey",
    DestinationBucket = bucketName,
    DestinationKey = "/destkey",

    DisableTrimmingLeadingSlash = true
})

Our previous approach was not acceptable as we found use cases that depended on the existing behavior (which we understand is inconsistent, but we cannot break their workflows without a major version change).


Rename addon tables, reset migrations

Purpose
It seems django only uses app_name.split('.')[-1] as the prefix for tables unless you set app_label. So I set app_label and recreated the addon migrations.
Changes

Side effects

Ticket

Fixed on local @icereval @cslzchen :
                            List of relations
 Schema |                       Name                       | Type  | Owner
--------+--------------------------------------------------+-------+-------
 public | addons_box_nodesettings                          | table | osf
 public | addons_box_usersettings                          | table | osf
 public | addons_dropbox_nodesettings                      | table | osf
 public | addons_dropbox_usersettings                      | table | osf
 public | addons_osfstorage_nodesettings                   | table | osf
 public | addons_twofactor_usersettings                    | table | osf
 public | auth_group                                       | table | osf
 public | auth_group_permissions                           | table | osf
 public | auth_permission                                  | table | osf


Help file layout is inconsistent with rest of Max

Help files currently have the full helpdetails.js at the top of the patchers. The first tab should have this, with the digest stripped out for subsequent patchers.
Check groove~ for a reference.

There's also a quite different overall "style" with loads of colors and boxes and such, vs vanilla/native objects (fluid.dataset~ is perhaps a decent example).

This is largely taken care of with the various revisions of help files that have been going on this week.


Frame screenshot

Can't find how to hide "Take screenshot" in frame. For example:
https://www.w3schools.com/html/html_iframe.asp

Also this in tab-context.css is not working:
#tabContextMenu #context_sendTabToDevice,                  /* Send Tab to Device      */
#tabContextMenu #context_shareTabURL,                      /* Share tab URL via 3rd party app */

I still have "share" button:

Firefox 92.0.1

Thanks for the report. Probably the selectors changed or I simply forgot the new sub-menu.
I will look into it on saturday.

"Take Screenshot" is new for frame contextmenu. I added it.
https://github.com/stonecrusher/simpleMenuWizard/commit/329dab5bd92b7545cda9b7a766a89a33ed33786c
/* #context-take-frame-screenshot,          /* Take Screenshot          */
/* #context-take-frame-screenshot
                       + menuseparator,     /******** Separator *********/

Unfortunately I cannot see a "share" item anymore. Maybe only if you're logged in?

Send Tab To Device is still in the source code (#context_sendTabToDevice), but hidden for me. The selector seems unchanged.
Not sure what they changed. I'm also on FF 92.0.1

Still can't see "share" with a fresh profile, but can see it in another one. Fixed.
No news about #sendTabToDevice. Please open a new issue if necessary.
https://github.com/stonecrusher/simpleMenuWizard/commit/424b6e483d6f00018714d3ca67ec1dc0de54bdde


Restore aop.Select behavior for CloneModuleAsRecord

It turns out the behavior of Select differs from what we discussed in dev today, it actually returns the same module multiple times when you have CloneModuleAsRecord clones. Perhaps this isn't what we want, but I decided to add a test (that passed prior to https://github.com/chipsalliance/chisel3/pull/1974 and failed after it).
We could filter out these duplicates, but I wanted to document the behavior and discuss on this PR.
Contributor Checklist

[NA] Did you add Scaladoc to every public function/method?
[x] Did you add at least one test demonstrating the PR?
[x] Did you delete any extraneous printlns/debugging code?
[x] Did you specify the type of improvement?
[ ] Did you add appropriate documentation in docs/src?
[x] Did you state the API impact?
[x] Did you specify the code generation impact?
[x] Did you request a desired merge strategy?
[NA] Did you add text to be included in the Release Notes for this change?

Type of Improvement

bug fix

API Impact
Fixes aop.Select behavior for CloneModuleAsRecord
Backend Code Generation Impact
No impact
Desired Merge Strategy

Squash

Release Notes
Reviewer Checklist (only modified by reviewer)

[ ] Did you add the appropriate labels?
[ ] Did you mark the proper milestone (3.2.x, 3.3.x, 3.4.x, 3.5.0) ?
[ ] Did you review?
[ ] Did you check whether all relevant Contributor checkboxes have been checked?
[ ] Did you mark as Please Merge?

I don't want to merge this until https://github.com/chipsalliance/chisel3/pull/1985 is merged


Preset Cube causes error

When reproducing the snack, and using the cube's preset I get the following error:
Invariant Violation: Transform with key of "perspective" cannot be zero: {"perspective":0}
Looking for workarounds/fixes.
Thanks!

Any update on this?

@computerjazz


user name field from setting BLOG_USERNAME_FIELD in AuthorEntriesView

Current coverage is 92.37% (diff: 100%)

Merging #19 into develop will increase coverage by <.01%

@@            develop        #19   diff @@
==========================================
  Files            27         27
  Lines          1403       1404     +1
  Methods           0          0
  Messages          0          0
  Branches        173        173
==========================================
+ Hits           1296       1297     +1
  Misses           57         57
  Partials         50         50

Powered by Codecov. Last update ab4b602...9e55be7


Build Failed In ubunutu.

Hi everyone,
Iam new in react-native but i have already use react.js im using UbuntuOS i have installed node jdk8 watchman and everything but i am getting error build failed i have install adb and my phone is connect and adb devices showing my phone list but i dont know why error comin
Error: com.android.ide.common.perocess.ProcessException: Failed to execute aapt
please help me

I'm sorry you're running into this. I highly advise updating your post to use the template. We need to know more - specific React Native version you're using, all the steps you've followed, and what you have tried in order to debug this yourself, for example.


The configuration of user_default_llm in the service_conf.yaml configuration file does not take effect.

Is there an existing issue for the same bug?

[X] I have checked the existing issues.

Branch name
master
Commit ID
1
Other environment information
The configuration of user_default_llm in the service_conf.yaml configuration file does not take effect.

Actual behavior
https://github.com/infiniflow/ragflow/issues/586
Expected behavior
https://github.com/infiniflow/ragflow/issues/586
Steps to reproduce
The configuration of user_default_llm in the service_conf.yaml configuration file does not take effect.

Additional information
The configuration of user_default_llm in the service_conf.yaml configuration file does not take effect.

Upgrade to the latest dev version of the docker image.
For newly signed up account, this might work.


[action] plugin_scores: Revert support for rendering multiline description of plugin actions on docs

Reverts fastlane/fastlane#15212 due to regression spotted at https://github.com/fastlane/docs/pull/864.
To not block future fastlane releases and being able to update the docs with each release, this PR reverts the docs regression introduced by multiline description support in this PR.
https://github.com/fastlane/fastlane/issues/15201 should be re-opened once we merge this one.
🙇

@janpio very well, working on fix for plugin description generation 👍

Closing in favour of https://github.com/fastlane/fastlane/pull/15266.


Horizontal Stack

Hi, I have a problem, when I add to horizontal stack ptz cameras with webrtc,  ptz function not working. Any ideas ?
I need multiple cameras in one line.
 - type: custom:button-card
        icon: mdi:cctv
        tap_action:
          action: call-service
          service: browser_mod.popup
          service_data:
            title: Kameros
            hide_header: true
            large: true
            card:
              type: horizontal-stack
              cards:
                - type: custom:webrtc-camera
                  entity: camera.sonoff_camera
                  ptz:
                    opacity: 0.4
                    service: sonoff.send_command
                    data_up:
                      device: 'xxxxxx'
                      cmd: up

You can reopen the issue if the problem is still relevant


Add Finch support

Still need HIDAPI

Closed by 3a5eb6816638d308cf9b96aa00e2b99d56b019ac

Re-opening this, per @Zamua. Finch support may have never worked, and duplicating it into Ansible was insufficient.

export PYTHONPATH=$PYTHONPATH:/path/you/want/to/add Ref

Are there any more changes in particular that need to made for Spring or should this issue be re-targeted to Fall 2018? Or do we want to wait for comments on beta 1?

I'm ok retargeting this to fall. That doesn't preclude pushing fixes sooner, but I don't think there's the demand for this in the spring to be worth blocking the image release.

Do we need to look at the steps at the bottom of https://www.finchrobot.com/learning/compiling-python-programs

Looks like we already have libusb1.0.0. We probably want to avoid the script. It symlinks libudev.so.1 to to libudev.so.0. The udev rules in the zip have an extra line compared to the .rules file we have.

Re-targeting to Fall 2019. I am not convinced it would be useful to push this change so late in the semester (I believe the Finch projects have already begun and/or ended) and CS 101 will not be offered in Spring.

According to current students and TAs for CS 101, this has not been an issue this semester for students running the virtual machine. Closing.


Launch manual aggregation feature

This removes the devmode flag checks.
Tests: Manual, Unit
Bug: http://b/226565085

Same for iterators.


Master adding full tests

This PR refactors prepare_project.py a bit to allow unit testing of its logic. This PR also adds sample apps for each of the supported .NET Core runtime versions. These apps are fully functioning apps that can be deployed to GAE, used to verify the generated Dockerfile, etc...

Hi, all apps included do. I would use the clean-1.1 one since .NET Core 1.1 is the current branch for .NET Core and the more popular.


Support Flow type aliases

6to5 supports type annotations, which is awesome, but to be fully tolerant of Flow, there needs to be support for type aliases, as well: http://flowtype.org/docs/type-aliases.html.
My current workaround is to declare type aliases in separate .js files that are not transpiled by 6to5, but as shown in the type aliases documentation, they are designed to be able to be declared in the file that uses them.

Parsing type aliases is supported. this parses fine for instance. If you're running into syntax errors then it's definently a bug. Mind providing the line it's occurring on?

@sebmck Was this a recent update? I admit that it works in the playground, but locally I'm using 6to5 2.12.0 with --experimental --blacklist=useStrict --react-compat --include-regenerator --source-maps-inline.

Ah, it was fixed in 2.12.6: https://github.com/6to5/6to5/issues/513. My bad, thanks!


readme: drop forum for discussions

#skip-changelog

Codecov Report

Merging #887 (ace0860) into main (888679b) will increase coverage by 5.61%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #887      +/-   ##
==========================================
+ Coverage   89.94%   95.55%   +5.61%
==========================================
  Files           9        2       -7
  Lines         169       45     -124
==========================================
- Hits          152       43     -109
+ Misses         17        2      -15

Impacted Files
Coverage Δ

dio/lib/src/dio_event_processor.dart

dio/lib/src/tracing_client_adapter.dart

dio/lib/src/sentry_transformer.dart

dio/lib/src/sentry_dio_client_adapter.dart

dio/lib/src/sentry_dio_extension.dart

dio/lib/src/failed_request_interceptor.dart

dio/lib/src/breadcrumb_client_adapter.dart

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 888679b...ace0860. Read the comment docs.


Is it possible to make PolicyResult extendable?

Hi @leastprivilege
If you can make set accessors of Permissions and Roles properties of PolicyResult class as public,  then it would be easy for creating a custom implementation of IPolicyServerClient.

Is this for doing unit testing?

Hi @brockallen. I was just trying to plug-gin some custom implementations to work with PolicyServer local by creating a custom implementation of IPolicyServerClient. The idea is to get all my claims from our custom database and bake inside PolicyServerClient implementation so that no need to depend on json file. But that is only possible if the PolicyResult has public setter for Permissions and Roles properties.
I hope I was able to explain it better.

The idea is to get all my claims from our custom database and bake inside PolicyServerClient implementation so that no need to depend on json

I still don't follow - there are two independent things here

claims are input to the policy. You can create the claims in any way you want - IOW - if you want to load the input claims from a database, why not wrap the policy server client in a custom service that does the input claim loading.

policies define the mapping from input claims to roles and permissions. In the Local version this is using the configuration system (which btw is extensible - so you could load the polices from somewhere else than a json file).

If you need more advanced scenarios I would recommend using the commercial version (or fork Local and go from there)

@leastprivilege - I think @noufionline had valid requirement. This local version works really well. I have prepared local version with EF, but only one limit are internal setters on PolicyResult.  I've created fork from Local version and rework PolicyResult but it would be nice - install PolicyServer.Local as nuget package and extend with for example with EF implementation of local store. :) Please consider it.

OK - PR please.

Thanks!


@ToString over enum to incorporate enum field name

Tested on 1.18.2 and 1.16.18
@ToString
enum SampleEnum {
    SOME_FIELD;
}

Currently above enum is compiled into below code.
enum SampleEnum {
    SOME_FIELD;

 public String toString() {
        return "SampleEnum ()";
    }
}

So whenever i run toString on List I will get N "SampleEnum()" strings.
It should be compiled to:
enum SampleEnum {
    SOME_FIELD;

 public String toString() {
        return "SampleEnum (name())";
    }
}

or anything that indicates enums field name.

I suggest to use return "SampleEnum." + name() + "()";


Revert "Update monitoring per component enabled/disabled"

Reverts red-hat-data-services/rhods-operator#67

Sync upstream, before merging monitoring changes


record without matched schema

Hi,
I have a small question, in case when I consume records from Kafka and the record do not have a matched schema (for example because it was published with wrong schema), would I get any exception which I could catch and handle ? or the record would be simply ignored by the stream processor ?
Thanks,

You would get an exception.


getCountryByName Always return null

in your code the parameter countryName set toUpper, but in comparator none.
So bianarySearch always not found the search country

thanks for catching the bug we will fix this in the next release


add VS Code IDE DuckDB related tools to SQL Clients section (#21)

as described in https://github.com/davidgasquez/awesome-duckdb/issues/21.
Updated SQL Clients and IDE tools section in our fork for this PR to preview updated doc:
https://github.com/RandomFractals/awesome-duckdb#sql-clients-and-ide-that-support-duckdb
We've included our Premium Pro data tools extensions as well, as they provide direct integration with DuckDB SQL Tools in VS Code IDE for PRQL docs and various markdown documents some DuckDB users might be interested to explore.

@davidgasquez That's fine for now. DuckDB users using VS Code could benefit from PRQL Code Lens and new Markdown SQL Tools extension that are totally different tools and hook into PRQL and markdown document editors to run DuckDB SQL queries with VS Code SQL Tools.
Also, DuckDB Pro Tools is a separate extension, not just a premium version of the free DuckDB SQL Tools extension with advanced features, but we are sure users can discover them all from our updated tools docs that cover them more in detail with demos and examples.
Let's merge with your changes for now. Thanks!

Awesome! Thanks for the extra details!


FIX: Use different URI in Deploy Workflows

We were using the same URI to deploy to both the Performance Manager and Ingestion containers. When we would merge a PR that touched both projects, it would deploy two docker images to the same repo with the same tag, overwriting one of them. When the next step in the deploy process would go to the repo to pick up the image with the tag, it ended up deploying the same image to both containers, resulting in either two Ingestions running or two Performance managers running.
To fix this, I've added two secrets to the github repo to replace the DOCKER_REPO secret; one for ingestion and one for performance manager. In the github workflows, we now use those docker repo secrets. Additionally, I've added more tags that should appear alongside the git hash to describe the images, making it easier to identify what the images are and when they are from.
Asana Task: https://app.asana.com/0/1202848167169706/1203403213474705/f

LGTM


Switch to Self-Hosted Mempool

Good Morning.
I've been getting a lot of connection errors, and I would like to use my self-hosted instance of mempool.space to avoid rate limits.
I would like to use my self-hosted mempool.space to get the LastBlock and MempoolFees.
I've updated the code to use https://blockchain.info/ticker for the price, which works fine, but I replaced mempool.space with https://<IP_ADDRESS>:4081/api/blocks/tip/height I'm getting connection error. Same thing with https://<IP_ADDRESS>:4081/api/v1/fees/recommended. Both of these API work in a web browser on my computer, and I'm not sure why I'm seeing Connection Error for both on the OrangeClock after uploading the changes.
Additionally, it would be nice to have these updates as a backup or using mempool.space as a backup so I have two sets of APIs I can use in the even that I hit rate limits.

Same issue, on my side :-1:
I'll take a look at that

How long were you running OrangeClock before mempool.space started rate limiting you? I've been running it for about 24 hours and haven't seen any issues, but will leave it running indefinitely.
Regarding the API endpoints not working in the script despite working on the browser, this could be related to a missing/invalid certificate on your self-hosted mempool server. Have you gone through the process of generating a self-signed certificate?


Clasp open --webapp Broken

Expected Behavior
clasp open --webapp opens and prints a URL.
Actual Behavior
clasp open --webapp
? Open which deployment?                               @HEAD - AKfycbwk1UxuC3Qq39HREKquqp5XA9cW2YZhSvwEITr8WBe5
Missing required parameters: scriptId

Hi Grant, I am still experiencing this issue. Tried running yarn global upgrade clasp and then re-opening the terminal window to no avail. Is there anything else I can try here?

I think I need to republish clasp. It should work on master though.

using command clasp open --webapp, your real result is : Missing required parameters: scriptId.


SC-214195/reset tree creation filters

Summary: (https://github.com/chanzuckerberg/czgenepi/pull/1381)

What: reset tree creation filters
Ticket: sc214195
Env: rdev

Demos:

Notes:
Checklist:

[x] I merged latest <base branch>
[x] I manually verified the change
[x] I added labels to my PR
[ ] I tested in multiple browsers
[ ] I added relevant unit tests
[ ] I have notified others of changes they need to make locally (migrations, jobs, package updates, etc)

probably not necessary to fix in this PR, but for future work with this component, we probably want to start thinking about doing a minor refactor to break out each of the components in RadioLabel/index into their own files.


Support Phoenix.HTML 4.0

This pull request updates the library to work with Phoenix HTML 4.0.

The formatter changes have been automatically made by my code editor. I can remove those and leave the functional changes only if you prefer that @danhper


allowed linechart series to define their own x and y accessors

I have data that looks like this:
[
  {
    month: 0
    value1: 1234,
    value2: 4567
  },
  {
    month: 1
    value1: 7890,
    value2: 7410
  },...
]

I would like to be able to not have to preprocess it into separate series in order to graph value1 vs value2. This change allows a yAccessor or xAccessor to be defined in the series that would override the default selector. I have modified only the LineChart, but I imagine this would impact all of the charts to be done properly. I can modify the others as well if this is the right approach.

Thanks for the PR. Please note that we have already discussed different data formats several times in the past, so you may want to read those discussions/issues a bit to understand why we've decided on this data format.  This is a very common data format among JS chart libraries (Highcharts, NVD3, Vega chart specification) that I'm familiar with, and the rest of our API depends on having discrete series objects to store things like "series name" and eventually other series-related info.  We've already changed data formats one time, so I'm reluctant to change them again, or to support multiple different formats.
Also, it's not clear to me how your PR relates to the example data you show above.  Your PR changes look for an xAccessor and yAccessor in each array object (ie, in each series).  This would allow users to set different x or y accessors for each series.  Is that what you're wanting to do?
Because from the example data above, as well as your description, it looks like you're wanting to support a single array of objects, with each object key referencing a different series.  Supporting this data format would require a lot more changes that what's in this PR, even for just the linechart.
If you'd like to discuss it after you've read through our prior discussions on data formats, please feel free. I'm open to discussion but I'm unlikely to support multiple different formats.

After reading the previous discussions it seems that transforming the data during render is a better option.


Deploying the model for Inference

Hi,
Would you be able to provide python script that can perform inference over one of the pre-trained models with one of the datasets that the code/model has been validated on ?

Hi,
Could you post the actual error/stack trace that you're getting? Those should be in the right format, but I remember we needed to make some edits to the text strings to make them work with the new instructions (the old process was much more labor intensive). It's possible something didn't make it over to the public code, but I'll need to look into where it's currently going wrong.

Hi,
So I have been able to setup the pre-training pipeline following the instructions that you had given. I was trying to set up the model under the finetune config and got this error:
`(.venv) Vicky@Vigneshs-Macbook Polymathic_CP % python train_basic.py --yaml_config config/mpp_avit_ti_config.yaml --config finetune
Initializing data on rank 0
Initializing model on rank 0
Model parameter count: 7285884
Starting from pretrained model at ~/Polymathic_CP/weights/MPP_AViT_Ti
Traceback (most recent call last):
File "~/Polymathic_CP/train_basic.py", line 551, in
trainer = Trainer(params, global_rank, local_rank, device, sweep_id=args.sweep_id)
File "~/Polymathic_CP/train_basic.py", line 80, in init
self.restore_checkpoint(params.pretrained_ckpt_path)
File "~/Polymathic_CP/train_basic.py", line 205, in restore_checkpoint
self.model.module.unfreeze()
File "~/Polymathic_CP/.venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1695, in getattr
raise AttributeError(f"'{type(self).name}' object has no attribute '{name}'")
AttributeError: 'AViT' object has no attribute 'module'
`
Could you check to see whether the shared model weights are in the right format ? Or is it that I might have missed something.
Thanks,
Vignesh

Thanks. That worked a charm. I have done a fix for this and you can find it here: https://github.com/gitvicky/multiple_physics_pretraining/blob/csd3/train_basic.py

Was working on setting up the model for inference initially and noticed this unusual artefacts popping up on the model outputs. I have provided the visualisation of the model output and the expected model target here:

I had only slightly modified your train_basic file to set it up for inference across trained models. The inference code can be found here: https://github.com/gitvicky/multiple_physics_pretraining/blob/b4f39c11c87df098bc3d79f767f6746a325daf83/test_basic.py#L379

Thanks for finding this! Since this discussion happened offline, for anyone reading this later the early patching was the result of a bug in the bias copy for finetuning. This was fixed in 462d2992b74563c0325d2ec8baf8cd8e7952b0bc so closing the issue.


Add support for Step Function type of either STANDARD (default) or EX…

…PRESS
Description
Adding support to create either a Standard or Express step function.
Motivation and Context
Support was added in the AWS Terraform provider with version 3.27 to allow for express step functions to be created. This enables that change through this module.
Breaking Changes
None
How Has This Been Tested?
Created the example resources in a personal AWS account without issue.

Thanks @cojmcc for the PR!
v1.1.0 has been just released.


add sonic-ize particle transformer to CMSSW_14_1_0_pre0

PR description:
Modified files:
Configuration/ProcessModifiers/python/allSonicTriton_cff.py
RecoBTag/ONNXRuntime/python/pfParticleTransformerAK4_cff.py
Create files:
Configuration/ProcessModifiers/python/particleTransformerAK4SonicTriton_cff.py
RecoBTag/ONNXRuntime/plugins/ParticleTransformerAK4SonicJetTagsProducer.cc
PR validation:

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

Before submitting your pull requests, make sure you followed this checklist:

verify that the PR is really intended for the chosen branch
verify that changes follow CMS Naming, Coding, And Style Rules
verify that the PR passes the basic test procedure suggested in the CMSSW PR instructions

also: there's now another ParT version... https://github.com/cms-sw/cmssw/pull/44641
hopefully this one is faster to convert now that the basic process has been done once already


Completed Enhancement on changing current password admin/customer should get mail #3526

Enhancement:
Link: https://github.com/bagisto/bagisto/issues/3526

admin is not getting mail on changing password from my account section

now both customer & admin are receiving mails on reset password


Component: Develop Vanilla Footer component

User Story

As a [user role below]:
IBM.com Library developer

I need to:
work closely with the Business Analyst and designer to develop the Vanilla Footer

so that I can:
provide this component to developers using other frameworks than React

Additional information

Story within Storybook with corresponding knobs
Utilize Carbon
Functional specs issue (#1314  )
React version dev story (#1668)
Design QA testing issue (#1785)
Prod QA testing issue (#1786)

Acceptance criteria

[ ] Include README for the react pattern and corresponding styles
[ ] Minimum 80% unit test coverage
[ ] A comment is posted in the Design QA issue, tagging Wonil and Roberta, when development is finished
[ ] A comment is posted in the Prod QA issue, tagging Praveen and Chetan, when development is finished

Closing as this work was done at the end of 2019.  Will find the link to the dev story and add.


utils.h doesn't exist, but util.h does

Hi!
Am I using the appropriate version of arrayfire to get this fortran binding running? The compiler kept giving an error that af/utils.h didn't exist. I checked my installation and found that I have af/util.h (not plural). I changed the include file path in fortran_wrapper.cpp to util.h and now I'm not getting that error, but others.

This is from using an older version of arrayfire. Please use latest arrayfire and the master branch of the fortran wrapper.


Why use numpy?

Hello,
I'm no python expert but I'd love to know why we need to use numpy.sin rather than math.sin in some packages.
Added numpy to turf adds a faily big dependency.
It is the same for rtree. It adds considerable dependency for, so far, a single use in lineintersect. Even turfjs does not use any rtree library. Python rtree lib relies on a "heavy" libspatialindex library. I could not find a python only rtree library.
Thank you for your input.
Regards,
Pierre

Hi Pierre,
Thanks for your interest in our project.
I agree, the numpy dependency makes not soo much sense. I guess, we introduced it out of convenience and we were not very strict in our code reviews. 😅  We use numpy only for standard operations. Changing from numpy to math would reduce the dependencies and improve the code quality. You are right and your suggestion would be a better solution. Thanks for pointing it out, I will create an issue. :)
I also agree with the rtree library and the "heavy" libspatialindex library. I also don't like the library much, but I never had any problems with it in my work-related experience. I'm not aware of any other "production"-ready implementation in python. Until we find a more suitable implementation, we have to use this one.
I disagree. Turfjs uses an implementation of rtrees rtrees . Here is the link to the repo.
Again, thanks for your interest. We both are a bit slow at the moment with the implementation of missing libraries. I started a new job and drown in work tasks. If I get some idle time, I would continue working here.
Best, Steffen

Hi Pierre,
Thanks for your interest in our project.
I agree, the numpy dependency makes not soo much sense. I guess, we introduced it out of convenience and we were not very strict in our code reviews. 😅  We use numpy only for standard operations. Changing from numpy to math would reduce the dependencies and improve the code quality. You are right and your suggestion would be a better solution. Thanks for pointing it out, I will create an issue. :)
I also agree with the rtree library and the "heavy" libspatialindex library. I also don't like the library much, but I never had any problems with it in my work-related experience. I'm not aware of any other "production"-ready implementation in python. Until we find a more suitable implementation, we have to use this one.
I disagree. Turfjs uses an implementation of rtrees rtrees . Here is the link to the repo.
Again, thanks for your interest. We both are a bit slow at the moment with the implementation of missing libraries. I started a new job and drown in work tasks. If I get some idle time, I would continue working here.
Best, Steffen


My file went to binary without knowing what I did.  Please help.

Does this occur consistently?
Repro steps:

Action: azureStorage.editFile
Error type: Error
Error Message: cannot open azurestorage:/appliance-configs/SRV-SVAM-IKON.unity.config.json?resourceId%3D%2Fsubscriptions%2F3190d125-e525-4108-b9b6-9a8ed719d274%2FresourceGroups%2Fsnow-commerce-production%2Fproviders%2FMicrosoft.Storage%2FstorageAccounts%2Fstorageresortconfigprod%2FFile%20Shares%2Fappliance-configs. Detail: File seems to be binary and cannot be opened as text
Version: 0.9.0
OS: win32
OS Release: 10.0.16299
Product: Visual Studio Code
Product Version: 1.46.1
Language: en

Call Stack
file:/workbench.desktop.main.js:6011:738
processTicksAndRejections task_queues.js:85:5

@danSwiss can you confirm that the file isn't somehow corrupted? You can download it using the Azure portal to check this.

Right it was corrupted and we aren't sure how it occurred.  It gave this
option to report the issue.   Issue reports that the file can become
corrupted, please create a warning before saving that my file could become
corrupted when changing it incorrectly.
It is up to you how you want to proceed.  But I suggest you create a way to
lint the file before saving.
On Fri, Jul 24, 2020, 1:34 PM Will Lorey<EMAIL_ADDRESS>wrote:

@danSwiss https://github.com/danSwiss can you confirm that the file
isn't somehow corrupted? You can download it using the Azure portal to
check this.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode-azurestorage/issues/716#issuecomment-663699202,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AEYCZOOCGV2KWGYRAV7JKVTR5HO4RANCNFSM4PG3N2NA
.

@dannnyswiss thanks for the update. Just to verify - it sounds like you edited and saved the file in the extension before you saw this error. Or did the error occur when you opened the file for the first time?

I saw the error when I wanted to open the file for the first time.
From: Will Lorey<EMAIL_ADDRESS>Sent: Friday, July 31, 2020 12:26 PM
To: microsoft/vscode-azurestorage<EMAIL_ADDRESS>Cc: Daniel Swiss (DEN<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [microsoft/vscode-azurestorage] My file went to binary without knowing what I did. Please help. (#716)
NOTICE: This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender.
@dannnyswisshttps://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdannnyswiss&data=02|01|dswiss%40alterramtnco.com|f8c6513ca57f4b3ff1d108d8357f1fe0|bb43b24f11394bd9821c6cebb99a16cf|0|0|637318168006625499&sdata=jh1xsCVvieRw%2FB9q0OE3jTvN7C%2FxYJi%2FIAL3V7yuzFs%3D&reserved=0 thanks for the update. Just to verify - it sounds like you edited and saved the file in the extension before you saw this error. Or did the error occur when you opened the file for the first time?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmicrosoft%2Fvscode-azurestorage%2Fissues%2F716%23issuecomment-667286665&data=02|01|dswiss%40alterramtnco.com|f8c6513ca57f4b3ff1d108d8357f1fe0|bb43b24f11394bd9821c6cebb99a16cf|0|0|637318168006630489&sdata=3Gd%2FtsXAbBzlHfAwlCGFf7W0mGXLFMON0lTydOXY%2BDA%3D&reserved=0, or unsubscribehttps://can01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAEYCZONG6EOWCK4WYNEWFX3R6MECDANCNFSM4PG3N2NA&data=02|01|dswiss%40alterramtnco.com|f8c6513ca57f4b3ff1d108d8357f1fe0|bb43b24f11394bd9821c6cebb99a16cf|0|0|637318168006635479&sdata=1WqF2%2FXA8SE7Gu37zONPfTuStGcWSy5ugJnelYjxdLk%3D&reserved=0.

Potentially we could open files considered to be binary.  There seems to be a acceptTextOnly property in IReadTextFileOptions that is looked for before throwing this error.  That way even if a JSON is corrupt, we'd still be able to open it in the editor.
Related https://github.com/microsoft/vscode-azurestorage/issues/721

This issue has become stale and is at risk of being closed. The community has 60 days to upvote the issue. If it receives 5 upvotes we will keep it open and take another look. If not, we will close it. To learn more about how we handle issues, please see our documentation.
Happy Coding!

:slightly_frowning_face: In the last 60 days, this issue has received less than 5 community upvotes and we closed it. Still a big Thank You to you for taking the time to create it! To learn more about how we handle issues, please see our documentation.
Happy Coding!


[Bug] Intro side selection shows up a little early after initial Banapass detection

After going into the yellow screen comes up you can already interact with the side selector

How to replicate?

Ok saw it, can confirm this bug happened after me trying it

Noticed that the nameplate glow also appears very early

Fixed


[Feature] add ORL

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.
Motivation
Add ORL algorithm, including Python files and README.md
Modification
Please briefly describe what modification is made in this PR.
ORL is composed of three stages, which can be concluded as stage 1: Image-level pre-training and knn image ids retrieve,  stage 2: Roi_generate, stage 2: Roi_pair_retrieve and stage 3: object-level pre-training
[1] For dataset-related issues,
a. Regist 'SSDataset', 'CorrespondDataset', 'ORLDataset' in mmselfsup/dataset/init.py
b. ORL involves COCO train2017 dataset in stage 1 and stage 3. Add configs/selfsup/base/dataset/coco_orl_stage1.py, configs/base/dataset/coco_orl_stage3.py
[2] For Algorithm-related issues,
a. Regist "ORL, Correspondence, SelectiveSearch" in mmselfsup/models/algorithms/init.py
b. Add configs/selfsup/base/models/orl.py.
[3] For Hook issues,
Regist "ORLHook" in mmselfsup/engine/hook/init.py. ORLHook is adopted in stage 1 to retrieve knn image ids of COCO train2017
[4] For Bash Scripts,
a. ORL has three stages, therefore tools/slurm_train.sh and tools/dist_train.sh are not enough.
For stage 2: Roi_generate and stage 2: Roi_pair_retrieve, add four shell scripts, i.e.
tools/dist_selective_search_single_gpu.sh, tools/slurm_selective_search_single_gpu.sh,
tools/dist_generate_correspondence_single_gpu.sh, tools/slurm_generate_correspondence_single_gpu.sh,
b. For stage2: Roi_generate and stage2: Roi_pair_retrieve, add tools/generate_correspondence.py and tools/selective_search.py
[5] For pre-train configure file
Add configure files under configs/selfsup/orl/stage1/.py, configs/selfsup/orl/stage2/.py and configs/selfsup/orl/stage3/*.py
[6] Add README.md under configs/selfsup/orl/
BC-breaking (Optional)
Does the modification introduce changes that break the backward compatibility of the downstream repositories?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.
Use cases (Optional)
If this PR introduces a new feature, it is better to list some use cases here and update the documentation.
Checklist
Before PR:

[*] Pre-commit or other linting tools are used to fix the potential lint issues.
[ ] Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
[ ] The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
[ ] The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

[ ] If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects, like MMDet or MMSeg.
[ ] CLA has been signed and all committers have signed the CLA in this PR.

I have tried my best to transplant ORL to mmselfsup 1.x. Unfortunately, the pre-training of ORL is multi-stage and it needs thousands of epochs on COCO train2017 dataset, hence the provided computation resource is insufficient. As a consequence,  I hardly have the time and resources to reproduce downstream task results before DDL (1.16)


Dotp line graph in peak area replicate comparison.

Just add a comment on why this is there.
On Thu, Nov 18, 2021 at 4:21 PM Rita Chupalov @.***>
wrote:

@.**** commented on this pull request.
In pwiz_tools/Skyline/Controls/Graphs/SummaryBarGraphPaneBase.cs
https://github.com/ProteoWizard/pwiz/pull/1811#discussion_r752772937:

                   _graph.XAxis.Scale.Transform(X) / _graph.Rect.Width + OffsetPF.Width,

                   _yAxis.Scale.Transform(Y) / _graph.Rect.Height + OffsetPF.Height);

           }

       }

   }

   public virtual void PopulateTooltip(int index){}

   /// <summary>

   /// Override if you need to implement tooltips in your graph.

   /// </summary>

   /// <returns>A list of y-coordinates where tooltips should be displayed.

   /// List index is the replicate index.</returns>

   public virtual ImmutableList<float> GetToolTipDataSeries()

   {

       throw new NotImplementedException(@"Method GetToolTipDataSeries is not implemented.");

The problem is that if it's abstract then every non-abstract class that
inherits from SummaryBarGraphPaneBase has to implement it as well, even if
it has no intention to implement tooltips. I guess I can just make it to
return null, but I thought that explicit runtime error message is better
than an obscure null reference exception if this method is invoked by
mistake.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ProteoWizard/pwiz/pull/1811#discussion_r752772937,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACYBWUG2U5RFFOECLQSM2ATUMWJ77ANCNFSM5GUENAJQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.


Add a setting to switch to full vue build (with compiler)

Is your feature request related to a problem? Please describe.
Currently Vite is using runtime only build of vue (vue.runtime.esm-bundler.js)
Describe the solution you'd like
Having an option in vite config to switch between runtime only and full (with compiler) vue bundles.
Additional context
While it should be avoided, there are a number of usecases for compiling vue templates on the fly and it would be great if vue could support that. For example, we need a compiler build to run this simple example from vue website

With 0.20.6+ you can just do:
alias: {
  'vue': 'vue/dist/vue.esm-bundler.js'
}

@yyx990803 works with run dev, but once bundled with "yarn run build" dynamic components don't render.

Unsubscribe

On 08-Jun-2020, at 8:50 PM, Evan You<EMAIL_ADDRESS>wrote:
With 0.20.6+ you can just do:
alias: {
'vue': 'vue/dist/vue.esm-bundler.js'
}
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

@AXWELL911 please unwatch the repository if you don't want the emails. Don't use the reply as it spams everyone else.

@husayt please provide a reproduction in a separate issue.

This does not work in Vite 2.0.0-beta.27.
alias: {
  'vue': 'vue/dist/vue.esm-bundler.js'
}

I tried to find an option in https://github.com/vitejs/vite/tree/main/packages/plugin-vue, but I could not find an option there.
How to set a build with compiler?

This does not work in Vite 2.0.0-beta.27.
alias: {
  'vue': 'vue/dist/vue.esm-bundler.js'
}

I tried to find an option in https://github.com/vitejs/vite/tree/main/packages/plugin-vue, but I could not find an option there.
How to set a build with compiler?

Should be fixed by 54dab71, release soon

Should be fixed by 54dab71, release soon


New Version RuneLite 2.6.10

[ ] Have you signed the Contributor License Agreement?
[X] Have you checked that there aren't other open pull requests for the same manifest update/change?

[X] Does your manifest conform to the 1.5 schema?

Note: <path> is the name of the directory containing the manifest you're submitting.

This PR is the first RuneLite.RuneLite manifest that contains the arm64 installer, and also my first winget-pkgs contribution so I'm not sure if it will pass automatic validation.
There is an automatically generated open PR which also seeks to update the version to 2.6.10, but it contains the wrong URL for the x64 version, does not include the new architecture, has been stale for a while, and I don't have write access to modify the existing PR 😄
I also noticed at least the most recent existing version manifest incorrectly installs the x32 version on x64 PCs, which is what alerted me that there was an issue in the first place. That and there is a closed PR for the version before this, so I have at least one other PR to submit.
Microsoft Reviewers: Open in CodeFlow

@microsoft-github-policy-service agree


Creation of output panel slower

maybe related to https://github.com/Microsoft/vscode/issues/71366
check https://msit.powerbi.com/groups/2bb9c256-c8ed-47a3-b1be-ebf0c050fb85/reports/2f5bed71-2607-4e0c-8a83-d0f9de023342/ReportSection1f5ac8ab4f061b4a1f8d
output channels are restored slower

Output does not use the tree so it should not be the same cause.
Assigning to @sandy081

Yes, Symptoms and timelines look similar. Thanks @isidorn for digging it out.


Hyperlink is not working

\href{https://google.com}{Click Me}
This line is not working in compiled version
is there any other syntax?

Hello Ashu,
Please try this
\href {https://google.com}{\bf ClickMe}. I think this is the correct syntax.

\href{https://google.com}{Click}


Use swift-protobuf on ObjC+Swift project

Hi,
We are planning to use Protobuf on our project. Half of the code in the project are written in ObjC and one of our core data model was written in ObjC. I'm currently deciding whether we should use the ObjC Protobuf library or swift-protobuf. From what I can see, the generated code from swift-protobuf are structs and they don't seem very compatible to be used on Objective C. In that case, we will have to write Swift class wrapper around those generated structs in order to use them in our ObjC code.
Does anyone have any recommendation on this subject? We don't want to write wrapper for all the protobuf structs, will I be better off using ObjC version of the library?

You can always use the ObjC protos from Swift code (they just won't be value types).  So if you need to support that sort of a mixed world, it might make sense to just use the ObjC ones instead.
If you are going to migrate the ObjC code that needs protos to Swift at some point, then it might make sense to do wrappers to expose the Swift ones back to ObjC for the time being.

That makes sense. One thing that makes me hesitate to use the ObjC protobuf is the support of JSON. In some cases, we will pass back protobuf object in JSON format from the server, from I read so far on ObjC protobuf, it doesn't support JSON serialization currently (https://github.com/google/protobuf/issues/1675), does it mean the objc protobuf library does not support deserializing JSON format protobuf?
Let me know if I should post this question on the objc protobuf library instead. Thanks!

Correct, ObjC does not support JSON.  Since ObjC is mainly used for mobile, size of things really matters; so not having to the code or strings for JSON can be a savings.
The size concerns are also valid for Swift, but we're hoping the fact that Swift can do some dead stripping will help and we have #18 to look at making the strings needed optional so developers can decide to include/exclude the support.  It just wasn't something we went after for the 1.0 release.

Got it, another use case I'm thinking is for the ability of serializing/deserializing JSON is for persisting protobuf objects on disk. Do you have any recommendation on that? I have raised the question on the ObjC protobuf (https://github.com/google/protobuf/issues/3935)
Thanks for your help!

Replied over there.

I didn't a reply over there (or here), why do you feel JSON is the right path for persisting data?  The binary format is the one that actually handles unknown fields, so it much better suited to dealing with the message/enums/etc. evolving over time.

We agree that binary format is better for persistency, it's good to know that NSCoding is being supported in the ObjC Library. I believe Swift one also support NSCoding?

I believe Swift one also support NSCoding?

No, NSCoding these days slides into the whole Codable discussion; and since that includes its own definitions of a JSON encoding (and a binary one (via plists)), it can get really confusing quickly.  See some of the other issues (open and closed) that have asked about this.  It might make sense to support something like ObjC does (for NSSecureCoding and hence NSCoding), but given the potential confusion, the library does not do it at this time.

Interesting. I can see it can get complicated.
In that case, what would be the best way to persist a protobuf swift object?

Until the library provides something, you can always add your own extension to provide whatever helpers you need, I'd just use the binary serialization in the library for the actual format.

The result of serializing a protobuf type is a Data object, so you can just write that to a file:
try myObject.serializedData().write(to: ..., options: ...)

A good reason for supporting JSON, Text or any human readable format is debugging or mocking. Being able to persist protobuf objects on disk and modifying them by hand for testing, sniffing network traffic, these are very useful tools. Any updates on Text or JSON support for obj-c since the last discussion here?

The reasoning still stands (code size, etc.), for not supporting it in ObjC where no real dead stripping ever happens.  As far as persisting on disk, the textual formats really aren't as good for when the messages/enums may evolve with time, only the binary form is able to ensure old/new code can read old/new messages without losing anything.
fyi - this is the issue tracker for Swift, not ObjC.

Closing this out since the original question seems answered, and the one follow up since original opened was for ObjC, not SwiftProtobuf.


allMoves doesnt terminate

Reproduce:
ghci:> allMoves initialBoard W
or
ghci> length $ allMoves initialBoard W
appears for both black and white players.
last thing changed was to check for checks and filter allMoves whether you are moving into a check.
Probably somewhere there is a loop.
Also check if "moves" works and maybe if clearMaybeBoard has problems.
Lookout for additional tests.
This is also the reason why two tests fail (the agility fairness and the initialboard-tests)

Movement.hs
line 63 invokes allmoves for check
line 51 invokes check for allmoves
classic.


How to use run_squad.py to produce multiple answers for a question?

Hello,
I am using run_squad.py to build my own question answering system, the problem is that, I want the system can output multiple answers for a question.
The number of answers can be 0, or one, or multiple if possible, how can I do to the code to achieve this? Thank you

solved

@mushro00om can you please elaborate on how you did that?

@kshitij12345 Sure!
Some questions in my dataset have multiple answers, some have one answer, some no answer.
Firstly, I add a for loop in the "read_squad_example" method to allow the code to read all answers for each question and build N SquadExamples for each question, N is the number of answers (This is for my case, you don't have to do it, because I need to use all answers, the original squad code only reads the first answer of each question even the question has multiple answers).
The run_squad.py produces a "nbest_predictions.json" file, you can see the model provides top 20 possible answers for each question, with possibilities, so I just simply pick some of those answers according to their possibilities.
However, I have to admit that eventually the performance isn't that good. it works but just not that good, but I think it can be improved by some way.

Oh I see, Thank You.
Got it. I was wondering whether you were changing the head of the model to predict multiple answers.
Thank You again.

@mushro00om Hey, could you please share the code for getting more than one answers using BERT. It'll be really helpful for my application.

Hi,
Hahaha actually I didn't do anything special, once you have finished training, there will be a file created in the path (sry I forgot the name of the file, I think it's a text file), the file involves answers of each question and their probabilities (BERT will select the answer with highest probability as THE answer), what I simply did is just set a threshold, and then extract answers if the probability is greater than the threshold (for example 30%) as multiple answers. Just a very simple solution, hope it helps.
cheers

From: kmandhaniya<EMAIL_ADDRESS>Sent: Friday, 14 August 2020 4:07
To: google-research/bert<EMAIL_ADDRESS>Cc: mushro00om<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [google-research/bert] How to use run_squad.py to produce multiple answers for a question? (#657)
@mushro00omhttps://github.com/mushro00om Hey, could you please share the code for getting more than one answers using BERT. It'll be really helpful for my application.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/google-research/bert/issues/657#issuecomment-673684741, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALL77EYDM62CYKLXW46RWHTSARBYDANCNFSM4HOIRBPQ.

Oh, got it. Really helped me understand your logic. Thank you so much!


#32458 fix for of loop in internet explorer

Fixes #32458

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: Saulzi sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@Saulzi: I'm going to commit the two suggestions to this PR and once the build succeeds I'll go ahead and merge.

It looks like I'm unable to update the branch to commit these suggestions. @Saulzi are you able to take a look at this?

It looks like I can merge manually and make the minor fixes when I merge.

Thanks for the contribution!


Update IoT Central application limit

IoT Central allows users to create 100 applications per subscription by default.
This change updates the documentation where the older limit of 10 was referenced.

@emgarten : Thanks for your contribution! The author(s) have been notified to review your proposed change.

@dominicbetts,
Can you review the proposed changes?
IMPORTANT: When the changes are ready for publication, add a #sign-off comment to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

Hi @emgarten - thanks for this update. FYI, as an FTE you can contribute directly to the azure-docs-pr repo where the approval process is more streamlined.
#sign-off

@emgarten - here's some more information about working in the private repos:

Read about authors working in the public repo for more information.
If you need help getting set up to work in the private repo, see Set up Git repository locally for documentation.


misc. fixes for runtimes and packages

updated Lambda node runtime to something current
updated xml2js via resolutions

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

tested, working 👍


rename rational-complete module and use that namespace for company/autocomplete-mode configuration

What's wrong with rational-completion  as a name?

I'd put company into a different "development" configuration area, but I haven't decided the best way to organize that yet.


[SONARMSBRU-94] Changed verbosity of some messages

Adding missing logging to the properties fetcher.
Tweaked the logging of TeamBuildSettings so the verbosity is respected.

LGTM


runtime: exceeded thread wakeup limitation on iOS

We found my gomobile iOS application crash with the below message:
Event:           wakeups
Action taken:    none
Wakeups:         45001 wakeups over the last 64 seconds (704 wakeups per second average), exceeding limit of 150 wakeups per second over 300 seconds
Wakeups limit:   45000
Limit duration:  300s
Wakeups caused:  45001
Duration:        17.04s
Steps:           18

Hardware model:  iPhone8,1
Active cpus:     2
Boot args:

I guess Go runtime switched threads so many times that iOS complained, but I am not sure. It is hard to create a minimal code to reproduce this problem, but I'd like to report this (potential) problem in gomobile.

Please do try to figure out a way to reproduce the error; fixing the problem will be much more likely if you do.

I found a thread with a similar issue: https://forum.unity.com/threads/the-weird-ios8-unity3d-hiccup.273186/. It seems that Unity apps don't actually crash, only freezes while the exception report is generated.

Hey I want to work on this issue since I "translated" my app to go - and now I face problems here.
I found this thread at https://developer.apple.com/forums/thread/124180 and user eskimo has some insights about what to look for.

In short, it seems to count the number of times that a blocked thread was unblocked because of an ‘interrupt’

At first I tried to reproduce the issue in a minified example by doing thousands of Gosched() calls in multiple threads - unfortunately without success. My next attempt was mixing up some sync.Mutex .Lock() .Unlock() calls in multiple threads. This has lead to a lot of context switches in the Xcode profiler, however it has only produced reports for cpu_resource ("90 seconds cpu time over ...") but not the desired wakeups_resource report.
Now I'm kind of out of ideas what to look out for there, I'm trying to think of other design patterns that I use in my main project that could potentially fall into that "thread unblock or interrupt" territory but the Gosched and the Mutex calls are the only things I can come up with now. Maybe also channels but my understanding of these is quite limited
@hajimehoshi did make any further findings back then?
@eliasnaur has something like that ever been brought up again? Any info that could help me showcase a minimal example of the issue to debug or fix?

I'm afraid it is not an open source...

Oh too bad but don't worry about it, if at any time you remember something you stumbled across when debugging please let me know. And maybe Elias knows something too haha

It's the channels. My example project is quite messy, I am going minify my example project. So happy I found a way to reproduce :)

Please do try to figure out a way to reproduce the error; fixing the problem will be much more likely if you do.

@eliasnaur I have just made a project that reliably produces that error. https://github.com/g3devv/channelwakeups
It would be very exciting to hear from you if you have an idea how we could approach getting this resolved in gomobile

This is unlikely to be a gomobile issue, and more likely to be an iOS limitation that affects Go runtime scheduling. If you have a reliable reproducer, the Go runtime team may be able to help. I can come up with one clarifying question: is this still an issue on the very latest iOS version?
CC @golang/runtime .

Dear Elias, thank you for your response!
I completely agree that this very well may not be an issue with gomobile intrinsically, however I fear that some "specialties" that Go comes with make this issue more likely to come up in a Go app compared to native apps or apps developed with other frameworks.
What I mean by that (and I'm a complete novice when it comes to golang so maybe it's a bad example) is that for example I once read that Go is super fast at context switching and for that reason does it at a higher frequency than other language runtimes do.
So maybe, if we are able to find a singular "feature" to blame, it would be possible to pass a build flag (or compile flag or what it's called) to optionally disable (or throttle) such a feature; in a way that the generated framework is less likely at hitting these iOS restrictions.
Over the weekend, I was able to come up with a more concise example (5 lines of code) to reproduce the issue, which should make it easier to pin down the underlying cause. The error rate with the new code is still a bit temperamental (either because it's not hitting the nail OR iOS doesn't emit the crash info on each run). Once I was able to confirm my new demo is better, I will ping you again if that's okay with you.
About the iOS version, my production app showcased that issue on 15.2.1 and I currently test my demo project on 14.3 but I will update the phone tomorrow just to make sure.

You're welcome to ping me, but I'm unlikely to be able to do much more for this issue.

So maybe, if we are able to find a singular "feature" to blame, it would be possible to pass a build flag (or compile flag or what it's called) to optionally disable (or throttle) such a feature; in a way that the generated framework is less likely at hitting these iOS restrictions.

It's very unlikely an explicit flag will be accepted by the Go team, especially for a workaround for what seems to be a misfeature of a secondary port. What may be possible is that the Go runtime could somehow automatically limit thread switching when runtime.GOOS == "ios".

Ah, I see you provided a reproducer in https://github.com/g3devv/channelwakeups/blob/main/goproj/goprojlib/goprojlib.go.
I don't think that there is anything we can do here. If I am reading correctly, RunChannelConciseExample wakes every 4ms from sleep, which is 250 wakeups[1] per second. Even ignoring the rest of the program, this already far exceeds a 100 wakeup/s limit.
In a real application, the use of an unbuffered channel could exacerbate the issue because the producer side must sleep if the consumer isn't waiting for the next value. Adding a buffer would allow both sides to execute concurrently.
[1] Disclaimer, I don't know exactly what iOS considers a "wakeup", but I would certainly consider wake from sleep to be a wakeup.

This is kind of unfortunate, and we're not sure what to do about it. We could enforce coarse-grained wakeups on iOS, so that there isn't more than one wakeup every 10 ms (either spinning or waiting to wake up another thread).
The immediate workaround is to just have user application enforce this, but that can be difficult. It's surprising that iOS crashes the process instead of just refusing to wake up the process, though. That's a pretty hard hammer for what seems like a fixable problem.

CC @golang/ios

We're seeing this issue on macOS too, when running an app that links against a framework built with go.
It's extremely difficult to reproduce, and is only occurring on certain devices (of course, never our own).
The process is killed, and a diagnostic report is written to /Library/Logs/DiagnosticReports staying "Action taken: none", yet the process does appear to either crash or be killed by the OS.
Here's an excerpt from a diagnostic report when this happens:
Event:            wakeups
Action taken:     none
Wakeups:          45001 wakeups over the last 7 seconds (6142 wakeups per second average), exceeding limit of 150 wakeups per second over 300 seconds
Wakeups limit:    45000
Limit duration:   300s
Wakeups caused:   45001
Wakeups duration: 7s
Duration:         7.33s
Duration Sampled: 0.00s
Steps:            1

Hardware model:   Mac13,1
Active cpus:      10
HW page size:     16384
VM page size:     16384

Fan speed:        1328 rpm
Advisory levels:  Battery -> 2, User -> 2, ThermalPressure -> 0, Combined -> 2
Free disk space:  725.52 GB/3721.87 GB, low space threshold 3072 MB
Vnodes Available: 86.91% (228724/263168, 132392 allocated)

Preferred User Language: en-CA
Country Code:     CA
OS Cryptex File Extents: 1983

Heaviest stack for the target process:
  1  runtime.mcall + 84 (MyFramework + 3958644) [0x10369e774]
  1  runtime.park_m + 268 (MyFramework + 3785276) [0x10367423c]
  1  runtime.schedule + 152 (MyFramework + 3783896) [0x103673cd8]
  1  runtime.findRunnable + 1096 (MyFramework + 3777384) [0x103672368]
  1  runtime.stealWork + 440 (MyFramework + 3780744) [0x103673088]

Also to note, the framework is not built using gomobile, it's built directly with go build -buildmode=c-archive -ldflags '-s -w'

Some other folks appearing to run into a similar issue:
https://groups.google.com/g/golang-nuts/c/yywUWpRA1EE
https://gist.github.com/cvl/12989d329cc86e6f2490d14c8cbf3953

@iamcalledrob Thanks for the references. For posterity, the reproducer from https://groups.google.com/g/golang-nuts/c/yywUWpRA1EE is:
func main() {
    time.Sleep(10*time.Second) // added sleep to make sure it run enough to show in mac activities monitor
    for count < 10000 {
        count += 1
        netConn, err := dialer.Dial("tcp", ":8080")
        if err == nil {
            netConn.Close()
        }
    }

    time.Sleep(10*time.Second) // added sleep to make sure it run enough to show in mac activities monitor

}

@iamcalledrob does this program reproduce the problem for you as well?

@iamcalledrob what version of Go are you running? Can you paste go version and go env? Thanks!

The process is killed, and a diagnostic report is written to /Library/Logs/DiagnosticReports staying "Action taken: none", yet the process does appear to either crash or be killed by the OS.

FWIW, on https://developer.apple.com/forums/thread/124180?answerId=742350022#742350022 and following posts, it seems that this report is not supposed to actually kill the process, though, like you, the user seems to report that their process is dying.

Thanks for the additional details.
At a high level, I am still unsure what, if anything, we should do about this at the Go runtime level. The program that you shared is requesting 1 million goroutine wakeups per second (10k goroutines waking 100 times/s). So it doesn't seem fundamentally incorrect for the OS to warn about this.
The program in https://github.com/golang/go/issues/29284#issuecomment-1919785644 doesn't obviously contain sleeps, but 45k wakeups during 10k dials implies each dial sleeps at least 4.5 times on average. Perhaps we could do something about that?

Right, I totally get that the "reliable repro" is --a bit-- extremely contrived.
The challenge we are facing is that this "excessive wakeups -> crash" problem is occurring seemingly randomly in non-contrived production code. Code that isn't starting thousands of timers :)
I'm curious if there may be paths occasionally taken by the Go runtime that might be leading to an unexpectedly high number of wakeups?
We do a lot of network and disk IO, and these crashes are happening "in the wild" during these operations.

With the last iOS update I've seen virtually zero crashes to my app anymore. Maybe the crashes we've experienced came from another reason which added to the wakeup limitations or Apple has made changes on their end, or the thread wakeups have became less for some reason. But what I can tell is that it's not really an issue for us any more, it seemed to be a lot more critical when I first noticed it but now it's mostly fine ig :)


[Feature Request] Add environment variable to config_template to configure additional scopes for OpenID Connect

Is your feature request related to a problem? Please describe.
We want to configure OpenID Connect auth for Temporal UI and also enable default Authorizer / ClaimMapper in Temporal Server. The default claim mapper uses the "permissions" claim to extract namespace-level roles. So for OpenID we would need to add a "permissions" claim to the user's access token with e.g. "system:write" or "system:read" values.
The IDP we are using links claims to scopes, so we'd have to link the "permissions" claim to the "openid" scope. The openid scope is a global scope used by lots of other clients (not related to Temporal), so the "permissions" claim would be added to all these access tokens that don't really need it.
Instead we would like to be able to specify an additional custom scope via env variable
Describe the solution you'd like
Additional TEMPORAL_UI_OAUTH_SCOPE env variable that lets you specify an additional scope for OpenID Connect auth
auth:
  enabled: {{ default .Env.TEMPORAL_AUTH_ENABLED "false" }}
  providers:
  - label: {{ default .Env.TEMPORAL_AUTH_LABEL "sso" }}
    type: {{ default .Env.TEMPORAL_AUTH_TYPE "oidc" }}
    providerUrl: {{ .Env.TEMPORAL_AUTH_PROVIDER_URL }}
    issuerUrl: {{ default .Env.TEMPORAL_AUTH_ISSUER_URL "" }}
    clientId: {{ .Env.TEMPORAL_AUTH_CLIENT_ID }}
    clientSecret: {{ .Env.TEMPORAL_AUTH_CLIENT_SECRET }}
    callbackUrl: {{ .Env.TEMPORAL_AUTH_CALLBACK_URL }}
    scopes:
      - openid
      - profile
      - email
      - {{ .Env.TEMPORAL_AUTH_SCOPE "temporal_ui" }} // <------ NEW

Additional context

we can add a variable to list a number of scopes similarly to https://github.com/temporalio/ui-server/blob/618ee7303dd4070abcc2d836c0f868eec23c9915/docker/config_template.yaml#L50 if that works for you

Why should you have hardcoded scopes ? We try to configure the UI only using env variables for simplicity, and would like to be able to replace the default values with the scopes we need. I do not think that our auth provider ignores unsupported scopes, but I need to test that to be sure, and I'm still  bit away from that since there are other issues to solve too.

Agree, I think it is enough to have one variable to specify one or multiple scopes, no need to hardcode some openid-specific scopes.

following up in the new release asap


Ability to Group Channels

On the main page of Cumulus TV, I should be able to come up with my own Group Names and be able to group channels.  So currently you have "My Channels" and under that "Suggested Channels".  My suggestion is to have "All Channels", "Group 1", "Another Group"... "Suggested Channels".
Reason for this suggestion, I have a large list of IPTV channels, with some from Australia, and some from Indonesia.  It would be good to separate them, so if my partner wants to watch something from Indonesia, she doesn't have to scroll through all the Australian IPTV channels to get to the ones she watches.

Interesting idea, but low priority for me.
On May 20, 2016 7:38 PM, "Alan Cramer"<EMAIL_ADDRESS>wrote:

On the main page of Cumulus TV, I should be able to come up with my own
Group Names and be able to group channels. So currently you have "My
Channels" and under that "Suggested Channels". My suggestion is to have
"All Channels", "Group 1", "Another Group"... "Suggested Channels".
Reason for this suggestion, I have a large list of IPTV channels, with
some from Australia, and some from Indonesia. It would be good to separate
them, so if my partner wants to watch something from Indonesia, she doesn't
have to scroll through all the Australian IPTV channels to get to the ones
she watches.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/Fleker/CumulusTV/issues/120

When you have more than a hundred channels maybe its needed, you can do it by categorizite them, there is, movies, kids, science, etc.
The big problem for me its when I change channels, when I load my hundred channels list, if fails with most channels when Im tuning.
Is it there a way you could check this? The last build I was able to change all channels without problems was 1.4.7. but it took some seconds to change (more a minute sometimes) and sometimes it didnt change.

Sorting by genres may not be too complex.
Yeah I can take a look at the import mechanics. Im working on a new import
system now.
On Fri, May 20, 2016, 10:08 PM josuearisty<EMAIL_ADDRESS>wrote:

When you have more than a hundred channels maybe its needed, you can do it
by categorizite them, there is, movies, kids, science, etc.
The big problem for me its when I change channels, when I load my hundred
channels list, if fails with most channels when Im tuning.
Is it there a way you could check this? The last build I was able to
change all channels without problems was 1.4.7. but it took some seconds to
change (more a minute sometimes) and sometimes it didnt change.
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/Fleker/CumulusTV/issues/120#issuecomment-220759027

The problem for is not importing channels, it comes when changing channels.
I sent you a video of it before.
When will you release next version, the lastest its having some serious issues!

I'll be doing some debugging over the weekend. I have been getting a lot of
crash reports recently.
On Fri, May 20, 2016, 10:13 PM josuearisty<EMAIL_ADDRESS>wrote:

The problem for is not importing channels, it comes when changing channels.
I sent you a video of it before.
When will you release next version, the lastest its having some serious
issues!
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/Fleker/CumulusTV/issues/120#issuecomment-220759214

I wish Google themselves would allow this! I can't include Pluto in my Live Channels because it randomly gets placed in the middle of the other apps that I use more often meaning that sometimes I have to scroll through their almost 150+ channels to get to Cumulus (as it occasionally shows up as the package name and it uses lowercase after capital sorting)

There are filters in a navigation drawer or voice search to tune to a
particular channel.
On May 21, 2016 12:57 AM, "Brandon Giesing"<EMAIL_ADDRESS>wrote:

I wish Google themselves would allow this! I can't include Pluto in my
Live Channels because it randomly gets placed in the middle of the other
apps that I use more often meaning that sometimes I have to scroll through
their almost 150+ channels to get to Cumulus (as it occasionally shows up
as the package name and it uses lowercase after capital sorting)
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/Fleker/CumulusTV/issues/120#issuecomment-220764787

@Fleker I didn't think of Voice Search (but Voice Search is horrible with my lisp) and the nav filter is there but it's by genre or HD/SD quality and it would be cool of Google to add by provider so I can go to the sidebar and only show Cumulus channels or only Pluto ones

May you please just support the groups like IPTV Pro does?
Example:
#EXTINF:-1 group-title="GROUP NAME HERE",CHANNEL_NAME_HERE
http://example.org/stream.ts

It's low priority at the moment in terms of what I'm developing.
On Nov 1, 2016 10:47 AM, "Federico Zivolo"<EMAIL_ADDRESS>wrote:

May you please just support the groups like IPTV Pro does?
Example:
#EXTINF:-1 group-title="GROUP NAME HERE",CHANNEL_NAME_HEREhttp://example.org/stream.ts
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Fleker/CumulusTV/issues/120#issuecomment-257585556,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADI581T6q9t4Hf07u0g798G7hd_qmnr4ks5q51D_gaJpZM4Ijrtz
.

I don't get it... All the IPTV services have hundred of channels, how we are supposed to navigate trough them without any kind of organization?

I understand that it can be inconvenient. Although there are other issues I
also want to tackle and I only have a certain amount of time.
I'm willing to accept contributions from other individuals, although my own
contributions will be at a certain speed.
On Nov 1, 2016 11:27 AM, "Federico Zivolo"<EMAIL_ADDRESS>wrote:

I don't get it... All the IPTV services have hundred of channels, how we
are supposed to navigate trough them without any kind of organization?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Fleker/CumulusTV/issues/120#issuecomment-257596889,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADI582h6VprbuHyF1-rq2UmKrJlgnNqVks5q51pfgaJpZM4Ijrtz
.

I've made some headway in the most recent release. I'm not pulling the group name from imported m3u playlists yet.

Will there be a group setting added to the json file?
On 4 Nov. 2016 14:53, "Nick"<EMAIL_ADDRESS>wrote:

I've made some headway in the most recent release. I'm not pulling the
group name from imported m3u playlists yet.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Fleker/CumulusTV/issues/120#issuecomment-258342160,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADIfyrWudDrrV8smXpf0CpYD5-dfZ5iMks5q6rNJgaJpZM4Ijrtz
.

Right now I'm using the genres component
On Fri, Nov 4, 2016, 1:18 AM Alan Cramer<EMAIL_ADDRESS>wrote:

Will there be a group setting added to the json file?
On 4 Nov. 2016 14:53, "Nick"<EMAIL_ADDRESS>wrote:

I've made some headway in the most recent release. I'm not pulling the
group name from imported m3u playlists yet.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Fleker/CumulusTV/issues/120#issuecomment-258342160,
or mute the thread
<
https://github.com/notifications/unsubscribe-auth/ADIfyrWudDrrV8smXpf0CpYD5-dfZ5iMks5q6rNJgaJpZM4Ijrtz
.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Fleker/CumulusTV/issues/120#issuecomment-258347049,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADI589e6LfsT1w09DSrken6g3m2FeZ3Zks5q6sAhgaJpZM4Ijrtz
.

After seeing how this works, it is an improvement over the previous setup of having all the channels in one line.

@Fleker , I see the cumulus app shows the channels organized by group (gathered from the M3U plist) but in the Live TV app the groups aren't there. Are you working on it, is it a bug or what?

The live TV app isn't mine. It groups based on certain genres. Not a bug.
No fix planned.
On Feb 7, 2017 5:22 AM, "Federico Zivolo"<EMAIL_ADDRESS>wrote:

@Fleker https://github.com/Fleker , I see the cumulus app shows the
channels organized by group (gathered from the M3U plist) but in the Live
TV app the groups aren't there. Are you working on it, is it a bug or what?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Fleker/CumulusTV/issues/120#issuecomment-277957771,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADI586VbOS0qrErm5TrxJrG89E6unC5Yks5raEXigaJpZM4Ijrtz
.

I know it's not yours 🙂 but since it does support categories, I guess there must be a way to use them?

They are being used if you use the system genres.
On Tue, Feb 7, 2017, 3:59 PM Federico Zivolo<EMAIL_ADDRESS>wrote:

I know it's not yours 🙂 but since it does support categories, I guess
there must be a way to use them?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Fleker/CumulusTV/issues/120#issuecomment-278138080,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ADI58_kesWVh7fAIKam996VFX4fpdhfvks5raNsygaJpZM4Ijrtz
.

do you have a link where I can find more info? I don't think I understand

Docs will be added soon. Follow new issue for updates.


AP (mask) is 35.4

Loading and preparing results...
DONE (t=3.93s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *segm*
DONE (t=44.62s).
Accumulating evaluation results...
DONE (t=5.38s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.354
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.558
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.380
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.161
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.379
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.517
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.298
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.450
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.468
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.269
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.505
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.619
2019-05-09 06:57:34,621 maskrcnn_benchmark.inference INFO: OrderedDict([('bbox', OrderedDict([('AP', 0.3759153459368749)
, ('AP50', 0.5908149513831737), ('AP75', 0.407623512259985), ('APs', 0.21592651568910282), ('APm', 0.40564621888303226),
 ('APl', 0.4964606429195723)])), ('segm', OrderedDict([('AP', 0.35404836208584806), ('AP50', 0.558382551808117), ('AP75'
, 0.379614103783724), ('APs', 0.1613041010105462), ('APm', 0.37909971606652115), ('APl', 0.5174932965096802)]))])

Hi, I use the repo to train on train2017 and test on val2017 (8 GPUs), the backbone is resnet-r50. I does not change any other config, but the final AP (mask) is 35.4 which is lower 0.2 than 35.6, is it normal? Is this the normal performance variation?

I have not run on coco2017 before. I have run several times on coco2014, the variation is 0.1. I‘m not sure whether there is any difference between coco2014 and coco2017. Maybe you should run maskrcnn on train2017 to see how the results are, and then compare whether the maskcoring rcnn's improvement is normal or not.

2019-05-03 02:39:19,210 maskrcnn_benchmark.inference INFO: Preparing results for COCO format
2019-05-03 02:39:19,210 maskrcnn_benchmark.inference INFO: Preparing bbox results
2019-05-03 02:39:20,843 maskrcnn_benchmark.inference INFO: Preparing segm results
5000it [04:23, 20.66it/s]
2019-05-03 02:43:44,124 maskrcnn_benchmark.inference INFO: Evaluating predictions
Loading and preparing results...
DONE (t=1.57s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=42.52s).
Accumulating evaluation results...
DONE (t=7.35s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.378
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.594
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.412
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.216
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.410
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.495
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.312
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.490
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.514
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.326
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.550
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.651
Loading and preparing results...
DONE (t=3.18s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *segm*
DONE (t=48.47s).
Accumulating evaluation results...
DONE (t=5.80s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.342
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.561
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.362
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.155
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.369
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.505
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.293
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.448
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.468
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.271
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.504
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.623

I have trained maskrcnn on train2017 and tested it on val2017, the performance is normal.

My experiments show the variations of bbox AP and mask AP is both 0.1. Maybe the variation of 0.2 is normal. You can run a few more times to see whether the variation is normal.

Hi, I trained the model on a custom dataset. The training worked perfectly, but running python tools/test_net.py to evaluate the model gives me this error
loading annotations into memory...
Done (t=0.05s)
creating index...
index created!
2020-08-11 21:25:51,999 maskrcnn_benchmark.inference INFO: Start evaluation on 1042 images

Traceback (most recent call last):
  File "tools/test_net.py", line 94, in <module>
    main()
  File "tools/test_net.py", line 88, in main
    maskiou_on=cfg.MODEL.MASKIOU_ON
  File "/home/eldad/anaconda3/lib/python3.7/site-packages/maskrcnn_benchmark-0.1-py3.7-linux-x86_64.egg/maskrcnn_benchmark/engine/inference.py", line 379, in inference
    predictions = compute_on_dataset(model, data_loader, device)
  File "/home/eldad/anaconda3/lib/python3.7/site-packages/maskrcnn_benchmark-0.1-py3.7-linux-x86_64.egg/maskrcnn_benchmark/engine/inference.py", line 31, in compute_on_dataset
    output = model(images)
  File "/home/eldad/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/eldad/anaconda3/lib/python3.7/site-packages/maskrcnn_benchmark-0.1-py3.7-linux-x86_64.egg/maskrcnn_benchmark/modeling/detector/generalized_rcnn.py", line 51, in forward
    x, result, detector_losses = self.roi_heads(features, proposals, targets)
  File "/home/eldad/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/eldad/anaconda3/lib/python3.7/site-packages/maskrcnn_benchmark-0.1-py3.7-linux-x86_64.egg/maskrcnn_benchmark/modeling/roi_heads/roi_heads.py", line 43, in forward
    loss_maskiou, detections = self.maskiou(roi_feature, detections, selected_mask, labels, maskiou_targets)
  File "/home/eldad/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/eldad/anaconda3/lib/python3.7/site-packages/maskrcnn_benchmark-0.1-py3.7-linux-x86_64.egg/maskrcnn_benchmark/modeling/roi_heads/maskiou_head/maskiou_head.py", line 41, in forward
    x = self.feature_extractor(features, selected_mask)
  File "/home/eldad/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 489, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/eldad/anaconda3/lib/python3.7/site-packages/maskrcnn_benchmark-0.1-py3.7-linux-x86_64.egg/maskrcnn_benchmark/modeling/roi_heads/maskiou_head/roi_maskiou_feature_extractors.py", line 39, in forward
    x = torch.cat((x, mask_pool), 1)
RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 1. Got 5 and 20 in dimension 0 at /pytorch/aten/src/THC/generic/THCTensorMath.cu:83

Is there something wrong i'm missing. I used configs/e2e_ms_rcnn_R_50_FPN_1x.yaml

have you fixed it yet? i have the same problem with you.