Add support for resumable uploads for File API

Add support for resumable uploads by following the MediaFileUpload class in Python:
https://googleapis.github.io/google-api-python-client/docs/epy/googleapiclient.http.MediaFileUpload-class.html
Add some pydocs to clarify what the parameters specified are. Confirmed this is working locally:

https://paypal.me/AlexisL2483?country.x=MX&locale.x=es_XC


Validate null value in enums

Incorrect YAML
EnumObject:
  type: string
  enum:
    - NULL
    - ITEM1
    - ITEM2

Correct YAML
EnumObject:
  type: string
  enum:
    - 'NULL'
    - ITEM1
    - ITEM2

Issue: There are warnings if using the incorrect YAML in SwaggerHub, but swagger-cli said the document is valid.
Warnings from SwaggerHub: enum value should conform to its schema's type
Expect result: Same warning as SwaggerHub.

This issue moved to swagger-parser project #144


Write Unit test for UriResolverAggregator

Tests can be found here: https://github.com/polywrap/javascript-client/blob/origin-dev/packages/uri-resolvers/src/tests/aggregator-resolver/aggregator-resolver.spec.ts


Validate .app File Presence

I tried to use rebar3 on a repo with a rebar.config, but it wasn't OTP compliant (i.e. didn't have a .app or app.src file). Without the app file, I received an error. This pull request returns an error during app validation if the app file doesn't exist.
Error received without the fix below:
===> Uncaught error: {'EXIT',
                             {badarg,
                              [{erlang,length,[undefined],[]},
                               {lists,suffix,2,
                                [{file,"lists.erl"},{line,203}]},
                               {rebar_app_utils,consult_app_file,1,
                                [{file,"src/rebar_app_utils.erl"},{line,155}]},
                               {rebar_app_utils,load_app_file,2,
                                [{file,"src/rebar_app_utils.erl"},{line,134}]},
                               {rebar_otp_app,validate_app,2,
                                [{file,"src/rebar_otp_app.erl"},{line,85}]},
                               {rebar_prv_compile,build,2,
                                [{file,"src/rebar_prv_compile.erl"},
                                 {line,88}]},
                               {rebar_prv_compile,'-build_apps/2-fun-0-',2,
                                [{file,"src/rebar_prv_compile.erl"},
                                 {line,80}]},
                               {lists,foreach,2,
                                [{file,"lists.erl"},{line,1336}]}]}}

I'm not sure we should accept building a non-app. This is likely to break else where.

This doesn't add support for building a non-app. It just returns an error message saying the app file is missing when you try to use rebar on a non-OTP project.
My specific case was that I pulled down a repo that had a rebar.config file, but I didn't notice the app file was missing. I didn't realize it wasn't a true OTP app until I found the cause of the error rebar3 was throwing.

Oh, my bad! Thanks, this is perfect then.
Weird, it failed on 17 only... I'll rerun it, Travis is being slow and won't show me the error message.

Cool, builds now.

For posterity, here is the final error when running rebar3 compile in DEBUG mode:
===> Failed to read undefined for processing: missing_app_file


add complete err check

It is the cherry-pick of https://github.com/IBM/operand-deployment-lifecycle-manager/pull/659
Let's hold it until the release of 3.7.3

It is ready to merge, since 3.7.3 has been released.

@Daniel-Fan Please bump the operator version first before merging this PR

@horis233
Sure, the PR for bumping version is open https://github.com/IBM/operand-deployment-lifecycle-manager/pull/666


coverage: Shift to codecov in Travis CI.

@timabbott. To post the comments in the PR, you need to give access to Codecov using this.

Codecov Report

:exclamation: No coverage uploaded for pull request base (master@7e99262). Click here to learn what that means.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master    #5458   +/-   ##
=========================================
  Coverage          ?   86.45%
=========================================
  Files             ?      430
  Lines             ?    52960
  Branches          ?     2063
=========================================
  Hits              ?    45789
  Misses            ?     7171
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7e99262...c9ba60f. Read the comment docs.

you need to give access to Codecov using this.

Apparently, this is not needed.

Merged, thanks @umairwaheed!


Multiple server side rendering passes with different output create class name mismatch with some components

[x] This is not a v0.x issue.
[x] I have searched the issues of this repository and believe that this is not a duplicate.

Expected Behavior 🤔

Server should preload data using Apollo, render a CircularProgress while loading and then the content once the data is present. This was working in v3.x and is a pretty normal real-world use case.
Current Behavior 😯

There is a class name mismatch on the TextField/Avatar element. If I use a component other than TextField or Avatar (those are the ones I found so far anyway; Button is confirmed working without issue), or if the loading component is null, there is no error.
Steps to Reproduce 🕹

Link: https://codesandbox.io/s/l7j61qr96l?fontsize=14
The sandbox won't work obviously, due to the lack of a real API and server-side rendering, but this is the minimal example that causes the issue in my App.

Load data via a query and render a component while loading
Use a TextField/Avatar/potentially some other components in the real display output
Get a class name mismatch

Context 🔦

Your Environment 🌎

Tech
Version

Material-UI
v4.0.0-alpha.7

React
v16.8.3

ApolloClient
v2.5.1

Update: This also appears to happen if there is a list rendered using deferred query data.

Could you provide a minimale reproduction repository?

This is as minimal as I could get it, it's just a razzle boilerplate with the most minimal mui/apollo SSR setup
https://github.com/wingertge/mui-classname-bug

@wingertge This change solves the problem:
--- a/app/src/server.js
+++ b/app/src/server.js
@@ -36,7 +36,7 @@ server
         const sheets = new ServerStyleSheets()

         const markup = await getMarkupFromTree({
-            tree: sheets.collect(
+            tree: (
                 <ApolloProvider client={client}>
                     <ThemeProvider theme={theme}>
                         <StaticRouter context={context} location={req.url}>
@@ -46,7 +46,7 @@ server
                     </ThemeProvider>
                 </ApolloProvider>
             ),
-            renderFunction: renderToString
+            renderFunction: tree => renderToString(sheets.collect(tree))
         })

The problem with the first approach is that it render an extra circular progress that is not present on the client. The rendered tree doesn't match. The index doesn't match, the class names break.
We have few leverages. One is #, another one is #15140 other one is #6115.


Tests fail after merging with latest back-end master

The unit tests seem to be failing after merging with the latest master

Assigning to myself

Assigning to @bp323 for review

Coverage remained the same at 100.0% when pulling 466b864da78499773a76116eba190f21c7bac0bf on Coenego:issue-169-tests-fix into 506fb37a5d18c0eb0329906e97591d18d204e8a0 on CUL-DigitalServices:master.

Coverage remained the same at 100.0% when pulling 106084742fcf4bdb192139e736a8797208bbc34d on Coenego:issue-169-tests-fix into 506fb37a5d18c0eb0329906e97591d18d204e8a0 on CUL-DigitalServices:master.

Still a few issues left with the tests. Back to @Coenego

Coverage remained the same at 100.0% when pulling 7bd454a48d8823ff3da714d65a7b19192c28f557 on Coenego:issue-169-tests-fix into b98cb4fa6330f5b93748a5f3a7916271b93c8475 on CUL-DigitalServices:master.

Coverage remained the same at 100.0% when pulling 06449297c70d193a046804f9646785a7d760fd7b on Coenego:issue-169-tests-fix into b98cb4fa6330f5b93748a5f3a7916271b93c8475 on CUL-DigitalServices:master.

Reassigning to @bp323

Coverage remained the same at 100.0% when pulling 533d8fff6bcd1591bd00497a77be2d9dea5a82df on Coenego:issue-169-tests-fix into b98cb4fa6330f5b93748a5f3a7916271b93c8475 on CUL-DigitalServices:master.

Normal test users are now created using the tenant UI for the Casper tests. Assigning to @bp323 for review.

Coverage remained the same at 100.0% when pulling cbb83b2a7bdee240bf56fb619b71d3cbc2184c8d on Coenego:issue-169-tests-fix into b98cb4fa6330f5b93748a5f3a7916271b93c8475 on CUL-DigitalServices:master.

Verified and merged!


Handle attributes in mark-defun / beginning-of-defun.

Typical scenario where this can be useful:
I want to mark a test (of a type with a bunch of derive statements) to move some code around. Calling mark-defun (C-M-h) does not select the attribute (#[test] or #[derive(...)], respectively).
This tries to fix this state of affair.

r? @nikomatsakis
(rust_highfive has picked a reviewer for you, use r? to override)


README improvement

Improve readability of README file. Make text more concise and easier understandable?

Include warning at the beginning that some original function calls mentioned in the paper are deprecated and that users should follow exclusively follow the README.

Example snippets repeat alot of code, I think a general overview about what models are available and one example for how to call the extractor should be sufficient

The readme very intuitive! I have a few suggestions, but none of them are critical imo.

In "About the project": Consider changing "bunch of images" to "a directory of images" or similar
In "Functionality": "extract features for the THINGS image database" is implicitly true from the previous two bullet points. I guess you aim to say something like "Conveniently access features for >26,000 images in the THINGS image database" to emphasize convenience?
"use HDF5 datasets": Sounds like a technical detail, maybe it's sufficient to mention this in the docs? The rest of the readme seems to be more of a high-level overview.
Under "Basic usage" -> "module_name": Maybe you could mention - in a short sentence - what you mean by "module"? Laymen might not be familiar?
In the example call for AlexNet: Is it possible to make it more visually distinct, which parts are code and which parts are console output? Otherwise, users might think they have to manually specify that big chunk of output produced by extractor.show_model()
Acknowledgements: I assume this section is not finalized yet? :)

@oliver-contier could you add your review and make these changes (except for the final two points) in the PR?


matchDiffLines also matches removed lines

Describe the bug
Your example catch_deprecated_component (linked below) uses matchDiffLines to check a regex and when true mark the PR as using a deprecated component.
This function seems also to match removed lines in the Diff, which erroneously marks PRs that cleanup deprecated components.
To Reproduce
Steps to reproduce the behavior:

The .cm automation file

# -*- mode: yaml -*-
manifest:
  version: 1.0

automations:
  estimated_time_to_review:
    if:
      - true
    run:
      - action: add-label@v1
        args:
          label: "{{ calc.etr }} min review"
          color: {{ 'E94637' if (calc.etr >= 20) else ('FBBD10' if (calc.etr >= 5) else '36A853') }}
  {% for item in deprecated %}
  catch_deprecated_components_{{ loop.index }}:
    if:
      - {{ source.diff.files | matchDiffLines(regex=item.regex) | some }}
    run:
      - action: add-label@v1
        args:
          label: 'deprecated-component'
          color: '#FF0000'
      - action: add-comment@v1
        args:
          comment: |
            Usage `{{ item.old }}` is deprecated please use `{{ item.new }}` instead
  {% endfor %}

calc:
  etr: {{ branch | estimatedReviewTime }}

deprecated:
  - regex: r/translateByKey\(/
    old: translateByKey
    new: formatMessage
  - regex: r/addStyles\(/
    old: addStyles
    new: SASS in a separate file ([Component name].scss)
  - regex: r/addNotificationsFromQuery/
    old: addNotificationsFromQuery
    new: UI/hooks/useGetNotificationsFromQuery
  - regex: r/from 'enzyme'/
    old: Enzyme
    new: testing-utils/testingLibrary

The PR URL (as it contains the repo and PR identifiers): https://github.com/amboss-mededu/ui-amboss/pull/5281
Describe your PR relevant content: Replace deprecated function translateByKey

Expected behavior
We expected that this check only reacts to additions and not deletions
Screenshots

Additional context
{% for item in deprecated %}
  # Automation names must be unique, so this adds an iterator index to each instance
  catch_deprecated_components_{{ loop.index }}:
    # Triggered when any of the modified files use a deprecated component
    if:
      - {{ source.diff.files | matchDiffLines(regex=item.regex) | some }}
    # Apply a deprecated-component label, request changes, and post a comment with an explanation.
    run:
      - action: add-label@v1
        args:
          label: 'deprecated-component'
          color: '#FF0000'
      - action: request-changes@v1
        args:
          comment: |
            `{{ item.old }}` component is deprecated, use `{{ item.new }}` instead
  {% endfor %}

@deiga to catch only added lines (which is what you want), you shoule add ^[+].* at the start of the regex, so for example:
deprecated:
  - regex: r/^[+].*translateByKey\(/
    old: translateByKey
    new: formatMessage
  - regex: r/^[+].*addStyles\(/
    old: addStyles
    new: SASS in a separate file ([Component name].scss)
  - regex: r/^[+].*addNotificationsFromQuery/
    old: addNotificationsFromQuery
    new: UI/hooks/useGetNotificationsFromQuery
  - regex: r/^[+].*from 'enzyme'/
    old: Enzyme
    new: testing-utils/testingLibrary

This will work as the diff itself contains + at the begining of every added line and - at the begining of every removed line

@deiga you can also use this:
{{ source.diff.files | matchDiffLines(regex=['^[+].*', item.regex] | join) | some }}

then you dont need to change the list of items


auto switch of color theme based on browser API prefers-color-scheme

This PR is based on the work from https://github.com/microsoft/vscode/pull/86600

polish and refactoring
renamed  settings as

'window.autoDetectColorScheme'
'workbench.preferredLightColorTheme'
'workbench.preferredDarkColorTheme'
'workbench.preferredHighConstrastColorTheme'

auto switching happens when

event is received and 'window.autoDetectColorScheme' is true
a change to the preffered scheme is detected on startup and 'window.autoDetectColorScheme' is true
a preferred themed is changed and 'window.autoDetectColorScheme' is true

'window.autoDetectColorScheme' is currently off by default.

where's the santa hat


HANDESK CONFIG: NEW TICKET FROM EMAILS

Hi, everybody,
Im new handesk user. So i have a question, i think it is a silly one but i just cant figure out, so we’re going to appreciate if you could help us.
Well!, we were trying you awesome Helpdesk platform, which it is great, and fit to our needs. But after installing, and configuring it, we realize that when we send an email to our support email account, the Handesk does not create the new ticket.
I read: Email polling (new tickets and tickets updates) and we understand that this feature make possible that when a costumer send us an email to out support email the system create and open a new ticket or it isnt?.
Could you help us.
Thank in advance,

Yeah, that is exactly what it does,
you just need to setup your .env file with your email details and then add the cron to your server so it can be exectued every x minutes
* * * * * php /path-to-your-project/artisan schedule:run >> /dev/null 2>&1

Hi,
I appreciate you anwers, but it does not work.. below i pasted the config that i put into the  .env file, please could you take a look ant tell me what could it be wrong :
MAIL_HOST=mail.cccccc.com
MAIL_PORT=26
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_ENCRYPTION=null
MAIL_FETCH_HOST=mail.cccccc.com
MAIL_FETCH_PORT=110
<EMAIL_ADDRESS>MAIL_FETCH_PASSWORD=Abcdef2018/*/
MAIL_FETCH_OPTIONS=/pop3
MAIL_FETCH_USE_SSL=true
// for options check https://secure.php.net/manual/de/function.imap-open.php
and this is the cron job i config in my cpanel:

php  /home/cccccc/public_html/soportest/handesk/artisan/artisan.php  schedule:run >> /dev/null 2>&1

When i create and update a ticket via the principal panel the system send the email, the only thing that does not work or better: I cannot make it works: is the creation of ticket via the emails.
Regards.
Carlos.

Looks like this is a email configuration issue which I can't really help..
Try playing arround with the config values..


Big Data Analytics

Good afternoon Sir,
I am currently studying Big Data Analytics.
I have done hadoop mapreduce and apache pig and currently learning Scala
Can you please guide me about the scope of Big Data Analytics.

Is Data Science overrated?

Thank you sir....
When will you start Data Structures.

Thank you sir....


[SSR] Client data fetching in beforeMount mixin double-fetches

Moved from vue-ssr-docs#210.
Version
2.5.21
Reproduction link
https://github.com/phil294/vue-hackernews-ssr-alternative-data-fetching
Steps to reproduce
In SSR docs, option 2, Fetch data after the matched view is rendered, calls asyncData in beforeMount.
The pseudo-syntax is
if (asyncData)
  asyncData()

This does not check if asyncData was already called by the server. Thus, guided by the official docs, this leads to duplicate fetches. In the case of dynamic module registration, even to duplicate modules.
In the linked repro, I cloned vue-hackernews and changed the fetch method accordingly. I also added a console.log to when fetchIdsByType is called which happens when asyncData is called.
What is expected?
Data is fetched on server, client does not fetch it again (as is the case with the usual router.beforeResolve solution (step 1).
What is actually happening?
Data is fetched twice, also on client, because there is no check on whether the data is already there, effectively making option 2 unusable.

 Certainly the example could be improved in this way, but that should be solved in the repository for the SSR docs (linked from within said docs):
https://github.com/vuejs/vue-ssr-docs

@LinusBorg I had created this very issue @ ssr-docs repo already (210) where you adivced me to open it here instead. So according to you, there is no proper place to open this issue. I am confused. :-)
Regarding the issue itself: The usecase is: Prevent entry-client from calling asyncData hooks at page load. The data was already put into the DOM, there is no need to call asyncData in the frontend again; this would negate the meaning of SSR. I highly doubt this is business logic-dependent.
Now, in the docs, there are two approaches mentioned for client data fetching.

The "first approach" (as in HN demo etc) calls the asyncDataHooks in router.beforeResolve(...), thus preventing double-fetching.
The "second approach" (quoted in OP) calls the asyncDataHooks in beforeMount(...) via a global Mixin, which also happens on the initial route. This approach does not seem sane and needs to be wrapped somehow.

Should we reopen the issue at ssr-docs then?
Merry Christmas!

Sorry about the confusion, i must have misinterpreted your original issue in the docs repository. I've reopened it, let's continue there


Fix broken Open in Colab badges linked to example notebooks

Description
This issue aims to fix broken Open in Colab badges on docs.pybamm.org
Fixes #3817
Type of change
Please add a line in the relevant section of CHANGELOG.md to document the change (include PR #) - note reverse order of PR #s. If necessary, also add to the list of breaking changes.

[ ] New feature (non-breaking change which adds functionality)
[ ] Optimization (back-end change that speeds up the code)
[ ] Bug fix (non-breaking change which fixes an issue)

Key checklist:

[ ] No style issues: $ pre-commit run (or $ nox -s pre-commit) (see CONTRIBUTING.md for how to set this up to run automatically when committing locally, in just two lines of code)
[ ] All tests pass: $ python run-tests.py --all (or $ nox -s tests)
[ ] The documentation builds: $ python run-tests.py --doctest (or $ nox -s doctests)

You can run integration tests, unit tests, and doctests together at once, using $ python run-tests.py --quick (or $ nox -s quick).
Further checks:

[ ] Code is commented, particularly in hard-to-understand areas
[ ] Tests added that prove fix is effective or that feature works

I believe that the docs build in this PR is unrelated as it's not stable build. Nice to have @agriyakhetarpal review as well, I hope merging this to develop would fix the badges without having to update v24.1 branch.

Sadly, this won't fix the link failure for the 24.1 release. I wish there were a reliable method to test such links, since I don't think even #3387 would cover this scenario.

Would it be nicer to open a separate issue about this other than #3387 where we could also further discuss about exploratory ways to cover the issue?

Would it be nicer to open a separate issue about this other than #3387 where we could also further discuss about exploratory ways to cover the issue?

I think this issue is minor enough to say that it won't get repeated again

The badges are still not fixed for the stable version, should we cherry-pick this to v24.1?

The badges are still not fixed for the stable version, should we cherry-pick this to v24.1? or maybe wait for next rc

I surfed some links and I am understanding that RTD does build the documentation via GitHub webhooks from repository tags. So, yes, I think cherry-picking this commit to the v24.1 tag should fix this on the stable version of the documentation. I will leave this up to what @Saransh-cpp thinks about this, though my preference would be to wait until the next RC. Maybe Google Colab or https://githubtocolab.com changed something from their end, because I remember that this had worked when I had implemented it (which is why I kept it as tree instead of blob).

I won't suggest moving the tag around. The maximum we can do is create a v24.1post1 release, but I don't think that is required. Let's keep this for the next release.


Developer Workstation Deployment with Docker Guide link not working

Hi Team,
The link Developer Workstation Deployment with Docker Guide not working.

Document Details
⚠ Do not edit this section. It is required for developer.sitecore.com.com ➟ GitHub issue linking.

Page: Sitecore Experience Platform 10.4
Source: markdown/pages/downloads/Sitecore_Experience_Platform/104/Sitecore_Experience_Platform_104/index.md

Hi @AmitKumar-AK,
The document should be finalized and uploaded by the end of today (April 30).

I have confirmed that this link is now working.


Unexpected error output from Z3: unknown constant Prims.precedes@tok

The following code leads to the following error:
module Test32

assume val t: Type -> Type
assume val r (#a: Type) (w: t a) (x1: a) (x2: a) : Type0
assume val p (a: Type) : (w: t a{r w == precedes #a})

let lem () : Lemma (True) =
  assert (r (p nat) == precedes #nat)

Unexpected output from Z3 (error "line 127673 column 0: unknown constant Prims.precedes@tok")

Thanks for the report @jaylorch. precedes gets special treatment in the smt encoding, which also means that it is not set up for partial application reasoning or even typing in smt.
May be we shouldn't allow client code to program with it directly, e.g. by marking it private in prims.fst. This snippet would then fail to typecheck.
Another fix is to encode it completely, as we do with other symbols.
WDYT? cc @nikswamy and @tahina-pro

My personal preference would be to just improve the understandability of the error message. It would be nice if it explained the underlying issue, provided a workaround (writing a function wrapper around precedes, and using that), and perhaps even gave an example of using such a workaround.

...and it would be nice if the error happened at the time of encoding, rather than passing an invalid encoding to Z3.

For the record, I had mentioned to Jay offline that a workaround is to
module Test32

assume val t: Type -> Type
assume val r (#a: Type) (w: t a) (x1: a) (x2: a) : Type0
assume val p (a: Type) : (w: t a{r w == (fun x y -> precedes #a x y)})

let lem () : Lemma (True) =
  assert (r (p nat) == (fun x y -> precedes #nat x y))

Detecting this in the encoding as Jay suggests should be feasible. Changing the encoding to support partially applied precedes will be a fair bit harder and will likely introduce some noise in the overwhelmingly common case where precedes is fully applied.

Nik, Tahina, and I discussed it offline, and finally ended up adding support for partial applied precedes to the smt encoding. Closing the issue, thanks again Jay!


Fail to open a downloaded folder whose name ends with a dot

Storage Explorer Version: 1.31.0-dev
Build Number: 20230704.7
Branch: main
Platform/OS: Windows 10
Architecture: x64
How Found: From running test cases
Regression From: Not a regression
Steps to Reproduce

Expand one storage account -> File Shares.
Create a file share -> Create a folder named 'f1.'.
Download the folder -> Click 'Show in Folder' next to the activity log.
Try to open the downloaded folder.
Check whether succeeds to open the folder.

Expected Experience
Succeed to open the folder.
Actual Experience
Fail to open the folder.

Additional Context

Succeed to open the folder 'f1.', if folder named 'f1' exists. the contents will be the files in 'f1' not the files in 'f1.'.
This issue doesn't reproduce on Linux Ubuntu 20.04/MacOS Ventura 13.4.1 (Apple M1 Pro).

@richardMSFT Verified this issue on the main build 20240328.5. The behavior matches your comment.
In addition, one thing needs your confirmation. I tried to download a non-empty folder named 'f1.'. Still show '0' items transferred. Is this expected?


Setup for Out of Proc has invalid method definition

https://github.com/Azure/azure-functions-sql-extension/blob/main/docs/SetupGuide_DotnetOutOfProc.md#setup-for-input-bindings
public static async IEnumerable<Employee> Run(
this isn't valid - async methods must return a Task.

@MaddyDev Make sure you go through all the setup/samples in the guide and verify they're correct.


Stay logged in through multiple sessions

Email:
I have successfully added your Blazor OIDC middleware into my Blazor WASM app. I noticed that your middleware persists the tokens (access, refresh and ID) into the session storage. This therefore does not allow the user to stay logged in through multiple sessions. Is there a way to allow the logged in session to persist over multiple user sessions?
Solution:

Add configurable support for LocalStorage with https://github.com/Blazored/LocalStorage
Check that logout and single logout works

Perhaps this should be a toggleable feature, especially if the computer that is being used is shared.

Yes that is also a consideration.


Remove reference to Wiki on Voice and Tone page

Expected behavior
https://pharos.jstor.org/content-style-guide/voice-and-tone
Ends with "See the Wiki for the latest..." but we don't link to it, and this page is no longer up to date. Let's please remove the last two sentences on that page.
Screenshots or code

Duplicate of an existing Jira ticket


Update coroutine example, add walktrough

I can't stand behind the correctness of my explanation yet, so please review.
I also suspect it's more of a tutorial material, so separating into its own file could be useful.

Thanks a lot for your review, @disruptek, it's very helpful as you accurately point to places where my understanding still lacks.
The explanation of resume and jield could still to be improved further but it's better than nothing.


Attribute android:textIsSelectable="true" is not working

<org.sufficientlysecure.htmltextview.HtmlTextView
android:layout_width="match_parent"
android:layout_height="wrap_content"
android:textSize="16sp"
android:textIsSelectable="true"/>

textIsSelectable seems to be extremely buggy for TextViews in general (https://www.google.de/search?hl=de&q=textIsSelectable).

I never had a problem with the standard TextViews. Is there a workaround to make it works? I just need to be able to copy the selected text.
By the way thanks for the answer.

In my case the exact problem was that on Marshmallow and above, the text could be selected, but the text selection commands didn't appear above the textview.
I think the root cause is the overridden HtmlTextView.onTouchEvent() method, and that the mDontConsumeNonUrlClicks field set to true, because currently the method always returns false thanks to that boolean. If I set mDontConsumeNonUrlClicks to true, the text selection bubble appears above the text.
Currently there's no way to control that field outside the package (I modified the code of HtmlTextView to test my solution), but I would gladly make a pull request with a setter method. I'm just not sure what's the purpose of mDontConsumeNonUrlClicks currently, and whether it could be removed altogether or a setter would be better.

@ofalvai If you still have some free time I would appreciate a pull request. I think mDontConsumeNonUrlClicks can be removed!

Released in 3.4


feat(afhsb): remove afhsb

closes #1141
Summary:

Remove afhsb endpoint from acquisition, server, and clients

Prerequisites:

[x] Unless it is a documentation hotfix it should be merged against the dev branch
[x] Branch is up-to-date with the branch to be merged with, i.e. dev
[x] Build is successful
[x] Code is cleaned up and formatted

Thank you, I did a Ctrl + F for afhsb, but forgot I had a Python-only filter applied.


Smallish improvements

This PR features a bunch of smallish quality-of-life changes which might have minor functional and semantic consequences.
Andreas Lauser<EMAIL_ADDRESS>on behalf of MBition GmbH.
Provider Information

okay, I fixed the "mutable object used as default value" issue for EncodeState and DecodeState: https://github.com/mercedes-benz/odxtools/pull/163/commits/1e312f0b32da811da3bd0066869efff492aa13be . Unfortunately, this involved a bit of copying of DecodeState objects. (While quite inelegant, performance should actually slightly improve because the _replace methods of NamedTuple does basically the same and we now only need to copy the decode state once per function call instead of once per parameter...)

@andlaus
You forgot to update end of pdu field
https://github.com/mercedes-benz/odxtools/blob/c5954adf9d9354bdb844bb4c595f4e39a70b53ff/odxtools/endofpdufield.py#L127

thanks for the catch. I hope to have fixed this in #164 (https://github.com/mercedes-benz/odxtools/pull/164/commits/bdc56ee9dc763de6e6735f138c97dfa188d1dafe ). can you check if it works?


Hexagonal-Architecture

Evaluation article on hexagonal architecture by ( rameshelworthy )

No longer needed as the code is not in proper state


fixes fail test in ruby 2.5 by initalize warning

I test with ruby 2.5 and it fail by warning message.
The error is following:
F
=========================
     84:       logdev.close
     85:       $stderr, stderr = stderr, $stderr
     86:     end
  => 87:     assert_equal "log writing failed. disk is full\n", stderr
     88:   end
     89:
     90:   def test_close
/Users/btm/develop/ruby/chrono_logger/test/from_ruby_repo/test_logdevice.rb:87:in `test_write'
<"log writing failed. disk is full\n"> expected but was
<"/Users/btm/develop/ruby/chrono_logger/lib/chrono_logger.rb:79: warning: instance variable @pattern not initialized\n" +
"log writing failed. disk is full\n">

diff:
+ /Users/btm/develop/ruby/chrono_logger/lib/chrono_logger.rb:79: warning: instance variable @pattern not initialized
  log writing failed. disk is full
Failure: test_write(TestLogDevice)
=========================

I fixes it using defined? keyword before access @pattern.

Thank you for your pull request and sorry for late response! I will confirm the code later.

Sorry for my late response. 🙇
I have understood that the error is happened when running with -W option.


how can I make a cutom header on this v2 table ?

try renderHeader function: https://github.com/dwqs/v2-table#the-v2-table-column-component

thank you , this is very helpful ...


Added UserAgentFallback to App

Added UserAgentFallback to the App object

@Mandrakia  Thanks for your contribution.
Write me your postal address on Gitter and you will get Electron.NET sticker :)


consumer.c

Looking for assistance in initial testing of SSC / SMC mhVTL code to utilise tcmu-runner.
As a starting point I was looking at consumer.c
Any tips on how to see this backing store from either targetcli or manually using /sys/....
All I'm after from existing 'consumer' is to be able to respond to 'inquiry'.
Cheers

Just to make sure we are on the same page because it gets messy.
consumer.c is an example of a daemon that uses libtcmu to do some basic processing including getting/sending commands to the kernel, reading in devices setup in configfs and if needed handling netlink events.
tcmu-runner is the name of this project and also a daemon in this project that has handlers to convert SCSI commands to ceph, gluster, optical etc requests. It uses libtcmu for setup too.
If you are using consumer.c I am not sure if targetcli works. You might need to slightly change it so it does not always try to talk to a daemon named tcmu-runner or does not try to do some dbus calls. Normally people doing something like consumer.c have gone their own route and might be doing their own setup tools as well as daemon, so on this GH we do not really know what they do.
At the very least the format/steps you need to use in configfs is:

modprobe target_core_user
cd /sys/kernel/config/target/core
mkdir user_$N

here we must use "user" to tell the kernel we are using the target_core_user module, then $N must be a unique numerical id.

cd user_0

mkdir $device_name

Here device_name can be any user defined string to identify your device.

cd device_name

You can then write whatever you want to the control file with the dev_config argument for your daemon to use. The kernel will format that as:

tcm-user/<hba_num>/<device_name>/dev_config
tmcu-user is just a prefix used by the module.
hba_num is the $N in the user string from # 3.
device_name is the name from # 5.
dev_config is the string that comes after the dev_config argument.
For example for tcmu-runner and consumer.c you would do:
echo "dev_config=foo/any_other_variables_you_want_to_pass_the_daemon" > control
consumer's call to tcmulib_initialize will then loop over the devices you have setup in the kernel and setup them up for your daemon.
To setup the target part like iscsi, loop, fc, etc then you should read the targetcli/rtslib/lio docs. That setup is the same for tcmu and other backends like file and iblock.
Note that tcmu-runner the daemon has a sort of funky setup workflow in that targetcli/rtslib talks to configfs, configfs interacts with lio and the target_core_user module then that module does netlink to the daemon.
If you are not going to use targetlci/rtslib then you can have your daemon/tools interact with configfs directly and skip the netlink steps which some people do not like.

Thanks for the pointers.
I haven't decided what/how to manage configuration at this stage (as mhVTL already has an existing setup/config process) - Once I have something that compiles and responds to 'TUR' or 'INQUIRY', I think I'll be in a better place to decide what/how to handle device setup between tcmu-runner & mhVTL.
As a starting point, I thought I'd merge some of the op-codes and at least have something to show.
p.s. Is this the best place to ask questions - or is there an email/discussion forum anywhere ?

newbe question #2 :
Assume I finally get something to show up under /sys/class/uio/* (I did once but not been able to reproduce at the moment)..
How do I get some sort of initiator to connect to my target ?
Seeing my 'lab' is a laptop with virtual box - I'm assuming it's going to be iSCSI (but I'm not fussed what initiator is)

I have to admit I'm struggling to find a way to bind the /loopback to the user:foo /backstores/user:foo
Any pointers would be most welcome. (be it loopback or iSCSI - I'm not that fussy)

p.s. Is this the best place to ask questions - or is there an email/discussion forum anywhere ?

For userspace this is the best place.
For kernel, target-devel is the best place.

I have to admit I'm struggling to find a way to bind the /loopback to the user:foo /backstores/user:foo
Any pointers would be most welcome. (be it loopback or iSCSI - I'm not that fussy)

I am not sure what you need. Can you even make a iscsi target and create a LUN using one of the existing normal old backstore like iblock or file? Here is the basics: how to make a target, create a backstore, and map it to a LUN using targetcli:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/storage_administration_guide/index#osm-target-setup

Setup: I think I have this right now.. I end up with a class/uio/uio0
modprobe  target_core_user
./consumer
In another ssh:
[root@mhvtltcmu-localdomain markh]# cd /sys/kernel/config/target/core
[root@mhvtltcmu-localdomain core]# mkdir user_0
[root@mhvtltcmu-localdomain core]# cd user_0
[root@mhvtltcmu-localdomain user_0]# mkdir foo
[root@mhvtltcmu-localdomain user_0]# cd foo
[root@mhvtltcmu-localdomain foo]# echo "dev_config=foo/mhvtl-device=10" > control
[root@mhvtltcmu-localdomain foo]# echo -n 1 > enable
[root@mhvtltcmu-localdomain foo]# cd /sys/class/uio/
[root@mhvtltcmu-localdomain uio]# ls -l
total 0
lrwxrwxrwx. 1 root root 0 Oct 25 09:09 uio0 -> ../../devices/tcm_user/uio/uio0
[root@mhvtltcmu-localdomain uio]# cd uio0/
[root@mhvtltcmu-localdomain uio0]# ls -l
total 0
-r--r--r--. 1 root root 4096 Oct 25 09:09 dev
lrwxrwxrwx. 1 root root    0 Oct 25 09:09 device -> ../../../tcm_user
-r--r--r--. 1 root root 4096 Oct 25 09:09 event
drwxr-xr-x. 3 root root    0 Oct 25 09:09 maps
-r--r--r--. 1 root root 4096 Oct 25 09:09 name
drwxr-xr-x. 2 root root    0 Oct 25 09:09 power
lrwxrwxrwx. 1 root root    0 Oct 25 09:09 subsystem -> ../../../../class/uio
-rw-r--r--. 1 root root 4096 Oct 25 09:09 uevent
-r--r--r--. 1 root root 4096 Oct 25 09:09 version
And 'targetcli' reports:
[root@mhvtltcmu-localdomain markh] # targetcli
targetcli shell version 2.1.fb49
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.
/> ls
o- / ......................................................................................................................... [...]
o- backstores .............................................................................................................. [...]
| o- block .................................................................................................. [Storage Objects: 0]
| o- fileio ................................................................................................. [Storage Objects: 0]
| o- pscsi .................................................................................................. [Storage Objects: 0]
| o- ramdisk ................................................................................................ [Storage Objects: 0]
| o- user:fbo ............................................................................................... [Storage Objects: 0]
| o- user:glfs .............................................................................................. [Storage Objects: 0]
| o- user:qcow .............................................................................................. [Storage Objects: 0]
| o- user:rbd ............................................................................................... [Storage Objects: 0]
| o- user:zbc ............................................................................................... [Storage Objects: 0]
o- iscsi ............................................................................................................ [Targets: 0]
o- loopback ......................................................................................................... [Targets: 0]
/>
The piece I'm struggling with is:
How to attach/bind/connect the user:foo object with an iSCSI target
Creating the iSCSI target is OK:
[root@mhvtltcmu-localdomain markh] # targetcli /iscsi/ create "iqn.2019-10.com.foo:target"
Created target iqn.2019-10.com.foo:target.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (<IP_ADDRESS>), port 3260.
I'll check out the above reading material..

I'm thinking my answers lay somewhere in the following :
http://linux-iscsi.org/wiki/ConfigFS
http://linux-iscsi.org/wiki/Target/configFS
If anybody knows off the top of their heads - feel free in jumping in :)

You just map the backend device you created to a LUN under the target. Check out the redhat.com doc above.

hmm... Still missing something here - sorry.
A snapshot of some of the options I've tried.. Some I wasn't expecting to work - but tried it anyway.
The ~/.targetcli/log.txt doesn't help either..
/> /iscsi/iqn.2019-10.com.foo:target/
/iscsi/iqn.20...om.foo:target> ls
o- iqn.2019-10.com.foo:target ............................................................................................ [TPGs: 1]
o- tpg1 ................................................................................................... [no-gen-acls, no-auth]
o- acls .............................................................................................................. [ACLs: 1]
| o- iqn.1994-05.com.redhat:e284d58153b4 ...................................................................... [Mapped LUNs: 0]
o- luns .............................................................................................................. [LUNs: 0]
o- portals ........................................................................................................ [Portals: 1]
o- <IP_ADDRESS>:3260 ......................................................................................................... [OK]
/iscsi/iqn.20...om.foo:target> cd tpg1/
/iscsi/iqn.20...o:target/tpg1> luns/ create /backstores/user:foo/uio0
storage object or path not valid
/iscsi/iqn.20...o:target/tpg1> luns/ create /backstores/user:foo/foo
storage object or path not valid
/iscsi/iqn.20...o:target/tpg1> luns/ create /backstores/uio0
storage object or path not valid
/iscsi/iqn.20...o:target/tpg1> luns/ create /backstores/uio/uio0
storage object or path not valid
/iscsi/iqn.20...o:target/tpg1> luns/ create /backstores/uio/uio0/
storage object or path not valid
/iscsi/iqn.20...o:target/tpg1> luns/ create
Missing required parameter storage_object
/iscsi/iqn.20...o:target/tpg1> luns/ create /backstores/uio
storage object or path not valid
/iscsi/iqn.20...o:target/tpg1> luns/ create /backstores/uio/uio0
storage object or path not valid
/iscsi/iqn.20...o:target/tpg1> luns/ create /backstores/user:foo/uio0
storage object or path not valid
/iscsi/iqn.20...o:target/tpg1> luns/ create /backstores/user:foo/uio
storage object or path not valid
/iscsi/iqn.20...o:target/tpg1> luns/ create /backstores/user:foo/foo
storage object or path not valid
/iscsi/iqn.20...o:target/tpg1> pwd
/iscsi/iqn.2019-10.com.foo:target/tpg1
/iscsi/iqn.20...o:target/tpg1>
$ cat /sys/class/uio/uio0/name
tcm-user/0/foo/foo/mhvtl-device=10

not sure if it makes any difference but:
targetcli --version
/bin/targetcli version 2.1.fb49

I think maybe you just want to use tcmu-runner and write a backend for that. Just look at file_optical.c. It is going to handle this stuff for you.
Also that targetcli version is really old. Just get the current version of targetcli-fb and rtslib-fb form the GitHub.com/open-iscsi repos.

Thanks for the tips..
Looked at installing the -fb but wasn't real sure what to do with the tar.gz (I'm sure I would have figured it out eventually)..
Went down the 'file_optical' route instead.
cp file_optical.c file_mhvtl.c (yeah - will fix later)
Updated CMakeLists.txt
/fbo/mhvtl/g -> file_mhvtl.c
make
make install
(OK, here I cheated and rebooted - I do need to identify the service to restart)
I now have a 'user:mhvtl' backstore :)
[markh@mhvtltcmu-localdomain tcmu-runner]$ sudo targetcli /backstores/ ls
o- backstores ................................................................................................................ [...]
o- block .................................................................................................... [Storage Objects: 0]
o- fileio ................................................................................................... [Storage Objects: 0]
o- pscsi .................................................................................................... [Storage Objects: 0]
o- ramdisk .................................................................................................. [Storage Objects: 0]
o- user:fbo ................................................................................................. [Storage Objects: 0]
o- user:glfs ................................................................................................ [Storage Objects: 0]
o- user:mhvtl ............................................................................................... [Storage Objects: 0]
o- user:qcow ................................................................................................ [Storage Objects: 0]
o- user:rbd ................................................................................................. [Storage Objects: 0]
o- user:zbc ................................................................................................. [Storage Objects: 0]
closing this - as I'll go down the 'file_optical.c' route instead of consumer.c


Allow options to be disabled

This change allows a consumer of the FormSelect component to disable options by passing a 'disabled' boolean value as part of the option.

Released in v5.1.0


[FEATURE]: support d1 kit push

Describe what you want
d1 now does not support kit push command.

This would be extremely valuable, right now I have to use something like this
import type { Config } from "drizzle-kit";

const driver = (process.env.DRIZZLE_DRIVER || "d1") as "d1" | "better-sqlite";

export default driver === "d1"
	? ({
			dialect: "sqlite",
			driver: "d1-http",
			schema: "./db/index.ts",
			out: "./db/migrations",
		} satisfies Config)
	: ({
			dialect: "sqlite",
			schema: "./db/index.ts",
			out: "./db/migrations",
			dbCredentials: {
				url: process.env.WRANGLER_DATABASE_URL as string,
			},
			tablesFilter: [`${prefix}*`],
		} satisfies Config);

and also manually find the wrangler sqlite file and run drizzle-kit push against it.

@steebchen this is cool, i'll try this out later

Should work well on latest, you can refer to this docs: https://orm.drizzle.team/docs/drizzle-kit-push#specifying-database-driver

Hmm it doesn't seem that local development is supported with this, or am I missing something? I'd like to be able to push against the wrangler dev db
pnpm push

><EMAIL_ADDRESS>push /Users/steebchen/projects/test
> pnpm drizzle-kit push --config drizzle.config.ts

Reading config file '/Users/steebchen/projects/test/drizzle.config.ts'
 Error  Please provide required params for D1 HTTP driver:
    [x] accountId: undefined
    [x] databaseId: undefined
    [x] token: undefined
 ELIFECYCLE  Command failed with exit code 1.

@AndriiSherman  still not work. should we merge #2941 to support this?


[Request] A simple way to bind component story state to args

Issue
I would like an easy way to have story state that is bound to the component but can still be controlled via args.
When writing stories for React it's very simple to have story state where the initial value is based on the story args, e.g.
const Template = (args) => {
  // Initial state is based on the args
  const [isOpen, setOpen] = useState(args.isOpen);

  return <Accordion {...args} isOpen={isOpen} onChange={setOpen} />;
};

export const Open = {
  render: Template,
  args: { isOpen: true },
};

export const Closed = {
  render: Template,
  args: { isOpen: false },
};

This allows you to have stories that 1) share a template, 2) show how the component reacts to different props, 3) are still fully interactive!
As a bonus, making it so the story state reacts to changes in the "Controls" panel is a bit more work but still straightforward:
const Template = (args) => {
  // Initial state is based on the args
  const [isOpen, setOpen] = useState(args.isOpen);
  // State updates when the args are changed
  useEffect(() => {
    setOpen(args.isOpen);
  }, [args.isOpen]);

  return <Accordion {...args} isOpen={isOpen} onChange={setOpen} />;
};

Reproducing this in addon-svelte-csf has proven difficult because I haven't found an easy way to access the story args in the initial render of the story. I've come up with a workaround (see below) but it feels very hacky and I would love to know if there's a built-in way to do it instead.
Example workaround
Note: This workaround has a limitation in that changes to state that happen inside the story code (e.g. form submission changing the loading state) will not update the story args, but this is the same as in my React example above and I think this is acceptable if not expected.
Form.svelte:

<form on:submit={handleSubmit}>

		Email address
		<input bind:value={email} id="email" type="email" disabled={loading} required />

	{#if error}

{error}

	{/if}
	<button type="submit" disabled={loading}>Submit</button>
</form>

Form.stories.svelte:

<Template>
	<Form bind:loading bind:error on:submit={handleSubmit} />
</Template>

<Story name="Default" />

<Story name="Loading" args={{ loading: true }} />

<Story name="Error" args={{ error: "We couldn't find an account with that email address." }} />

Okay so I dug in to the source code a bit more and actually ended up coming up with a much nicer workaround than the one in the OP (4 lines of code instead of 18). However I am still curious to know if this is indeed a supported use case and/or if there's a nicer way to do it.

<Template>
	<Form bind:loading bind:error on:submit={handleSubmit} />
</Template>

You should probably use an intermediate component to do that. I don't think it's a good practice to mix args with test states..


Conversion of .weights file to .pb file by darkflow

Hi,
I converted the .weights file trained using darknet into .pb file using darkflow and installed APK on Android cellphone, but fails to launch the application.
Is this conversion failed?
procedure:

$ cd ~/darknet
$ ./darknet detector train cfg/voc.data cfg/yolov2-tiny-voc.cfg bin/darknet19_448.conv.23
$ cd ~/darkflow
$ ./flow --module cfg/yolov2-tiny-voc.cfg --load bin/yolov2-tiny- voc-100.weights --savepb

result:
/home/keides2/darkflow/darkflow/dark/darknet.py: 54: UserWarning: ./cfg/yolov2-tiny-voc_100.cfg not found, use cfg / yolov2-tiny-voc.cfg instead
  cfg_path, FLAGS.model))
Parsing cfg / yolov 2 - tiny - voc.cfg
Loading bin / yolov 2 - tiny - voc - 100.weights ...
Successfully identified 63082056 bytes
Finished in 0.0044057369232177734s
Building net ...
Source | Train? | Layer description | Output size
------- + -------- + --------------------------------- - + ---------------
       | | input | (?, 416, 416, 3)
 Load | Yep! | Conv 3x 3 p 1 _ 1 + b norm leaky | (?, 416, 416, 16)
 Load | Yep! | Maxp 2 x 2 p 0 _ 2 | (?, 208, 208, 16)
 Load | Yep! | Conv 3x 3 p 1 _ 1 + b norm leaky | (?, 208, 208, 32)
 Load | Yep! | Maxp 2 x 2 p 0 _ 2 | (?, 104, 104, 32)
 Load | Yep! | Conv 3x3p1_1 + bnorm leaky | (?, 104, 104, 64)
 Load | Yep! | Maxp 2 x 2 p 0 _ 2 | (?, 52, 52, 64)
 Load | Yep! | Conv 3x3p1_1 + bnorm leaky | (?, 52, 52, 128)
 Load | Yep! | Maxp 2 x 2 p 0 _ 2 | (?, 26, 26, 128)
 Load | Yep! | Conv 3x3p1_1 + bnorm leaky | (?, 26, 26, 256)
 Load | Yep! | Maxp 2 x 2 p 0 _ 2 | (?, 13, 13, 256)
 Load | Yep! | Conv 3x3p1_1 + bnorm leaky | (?, 13, 13, 512)
 Load | Yep! | Maxp 2 x 2 p 0 _ 1 | (?, 13, 13, 512)
 Load | Yep! | Conv 3x3p1_1 + bnorm leaky | (?, 13, 13, 1024)
 Load | Yep! | Conv 3x3p1_1 + bnorm leaky | (?, 13, 13, 1024)
 Load | Yep! | Conv 1 x 1 p 0 _ 1 linear | (?, 13, 13, 30)
------- + -------- + --------------------------------- - + ---------------
Running entirely on CPU
2018-07-02 13: 26: 24.956143: I tensorflow / core / platform / cpu_feature_guard.cc: 140] Your CPU support instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2018-07-02 13: 26: 25.064272: I tensorflow / stream_executor / cuda / cuda_gpu_executor.cc: 898] successful NUMA node read from SysFS had negative value (-1), but there must always be at least one NUMA node, so returning NUMA node zero
2018-07-02 13: 26: 25.065139: I tensorflow / core / common_runtime / gpu / gpu_device.cc: 1356] Found device 0 with properties:
name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate (GHz): 1.759
pciBusID: 0000: 01: 00.0
totalMemory: 7.93GiB freeMemory: 7.22GiB
2018-07-02 13: 26: 25.065176: I tensorflow / core / common_runtime / gpu / gpu_device.cc: 1435] Adding visible gpu devices: 0
2018-07-02 13: 26: 26.025166: I tensorflow / core / common_runtime / gpu / gpu_device.cc: 923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-07-02 13: 26: 26.025209: I tensorflow / core / common_runtime / gpu / gpu_device.cc: 929] 0
2018-07-02 13: 26: 26.025235: I tensorflow / core / common_runtime / gpu / gpu_device.cc: 942] 0: N
Finished in 2.7535111904144287s
Rebuild a constant version ...
2018-07-02 13: 26: 26.655822: I tensorflow / core / common_runtime / gpu / gpu_device.cc: 1435] Adding visible gpu devices: 0
2018-07-02 13: 26: 26.655875: I tensorflow / core / common_runtime / gpu / gpu_device.cc: 923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-07-02 13: 26: 26.655886: I tensorflow / core / common_runtime / gpu / gpu_device.cc: 929] 0
2018-07-02 13: 26: 26.655896: I tensorflow / core / common_runtime / gpu / gpu_device.cc: 942] 0: N
2018-07-02 13: 26: 26.663997: I tensorflow / core / common_runtime / gpu / gpu_device.cc: 1435] Adding visible gpu devices: 0
2018-07-02 13: 26: 26.664021: I tensorflow / core / common_runtime / gpu / gpu_device.cc: 923] Device interconnect StreamExecutor with strength 1 edge matrix:
2018-07-02 13: 26: 26.664029: I tensorflow / core / common_runtime / gpu / gpu_device.cc: 929] 0
2018-07-02 13: 26: 26.664034: I tensorflow / core / common_runtime / gpu / gpu_device.cc: 942] 0: N
Created TensorFlow device (/ job: localhost / replica: 0 / task: 0 / device: GPU: 0 with 6972] Created TensorFlow device MB memory) -> physical GPU (device: 0, name: GeForce GTX 1080, pci bus id: 0000: 01: 00.0, compute capability: 6.1)
Done
(tensorflow) [keides2@bslpc ~/darkflow] $

Hi,
Could you please copy the exception which you get on the Android device?
You also can try out the retrained model with this demo application: https://github.com/szaza/tensorflow-java-examples-spring.

@szaza Thank you for your response.
Unfortunately, instead of throwing an exception, the application will be closed as soon as it is indicated that "YOLOv2 has stopped" at startup. Restarting the application is the same.

Hi,
Sorry for replying so late, I was very occupied in the last days.
Yes, theoretically it should be enough to copy the protobuff and labels file into the assets folder then rename them.
Are you sure that you do not get any exception in the terminal window when you run the application in debug mode with Android Studio?
Can you share the newly trained weights with me in private in order to check it (my mail address is: szabozoltanbors[at]gmail[dot]com)?
Have you tried to use the newly trained weights with the darkflow or darknet? Probably, it is a good idea to check its correctness before converting it into the protobuff file. Also a good idea to check it with darkflow after the conversion.

@szaza Sharing the weights file is fine.
Because the weights file used the data set of winnie-the-pooh and tiger listed in the Java API Example TensorFlow.
The weights file(zipped) is 56MB, so can I send it to you by e-mail?
I am able to detect the object from the image using the trained weights file.

./darknet detector test cfg/voc.data cfg/yolov2-tiny-voc.cfg backup/yolov2-tiny-voc_500.weights data/pooh-tiger/37.jpg

data/pooh-tiger/37.jpg: Predicted in 0.004631 seconds.
winnie-the-pooh: 46%
tiger: 87%
tiger: 63%
winnie-the-pooh: 44%
Not compiled with OpenCV, saving to predictions.png instead

I do not know how to detect an object from an image using the converted pb file.
I tried the command below, but in both cases an AssertionError occurred.
./flow --imgdir sample_img/ --model cfg/yolov2-tiny-voc.cfg --load built_graph/yolov2-tiny-voc.weights
./flow --imgdir sample_img/ --model cfg/yolov2-tiny-voc.cfg --load built_graph/yolov2-tiny-voc.pb

AssertionError: expect 63102556 bytes, found 63471556
I modified self.offset = 16 in loader.py to self.offset = 20, but it does not improve.
By the way, the following exceptions were occurring.
Failed to load model from 'file:///android_asset/tiny-yolo-voc-graph.pb'

07/05 09:55:11: Launching android-yolo-v2-master
$ adb shell am start -n "org.tensorflow.yolo/org.tensorflow.yolo.view.ClassifierActivity" -a android.intent.action.MAIN -c android.intent.category.LAUNCHER -D
Connecting to org.tensorflow.yolo
Waiting for application to start debug server
Waiting for application to come online: org.tensorflow.yolo | org.tensorflow.yolo.test
Connecting to org.tensorflow.yolo
Capturing and displaying logcat messages from application. This behavior can be disabled in the "Logcat output" section of the "Debugger" settings page.
D/TimaKeyStoreProvider: TimaKeyStore is not enabled: cannot add TimaSignature Service and generateKeyPair Service
W/ActivityThread: Application org.tensorflow.yolo is waiting for the debugger on port 8100...
I/System.out: Sending WAIT chunk
I/art: Debugger is active
Connected to the target VM, address: 'localhost:8601', transport: 'socket'
I/art: Starting a blocking GC Instrumentation
I/System.out: Debugger has connected
waiting for debugger to settle...
I/System.out: waiting for debugger to settle...
I/System.out: waiting for debugger to settle...
I/System.out: waiting for debugger to settle...
I/System.out: waiting for debugger to settle...
I/System.out: waiting for debugger to settle...
I/System.out: waiting for debugger to settle...
I/System.out: debugger has settled (1481)
I/InstantRun: starting instant run server: is main process
D/ViewRootImpl@230c34d[ClassifierActivity]: ThreadedRenderer.create() translucent=false
D/InputTransport: Input channel constructed: fd=75
D/ViewRootImpl@230c34d[ClassifierActivity]: setView = DecorView@128313[ClassifierActivity] touchMode=true
D/ViewRootImpl@230c34d[ClassifierActivity]: dispatchAttachedToWindow
D/ViewRootImpl@230c34d[ClassifierActivity]: Relayout returned: oldFrame=[0,0][0,0] newFrame=[0,0][1440,2560] result=0x27 surface={isValid=true<PHONE_NUMBER>36} surfaceGenerationChanged=true
D/ViewRootImpl@230c34d[ClassifierActivity]: mHardwareRenderer.initialize() mSurface={isValid=true<PHONE_NUMBER>36} hwInitialized=true
D/ViewRootImpl@230c34d[ClassifierActivity]: MSG_RESIZED_REPORT: ci=Rect(0, 0 - 0, 0) vi=Rect(0, 0 - 0, 0) or=1
D/ViewRootImpl@230c34d[ClassifierActivity]: MSG_WINDOW_FOCUS_CHANGED 1
D/ViewRootImpl@230c34d[ClassifierActivity]: mHardwareRenderer.initializeIfNeeded()#2 mSurface={isValid=true<PHONE_NUMBER>36}
D/libEGL: loaded /vendor/lib64/egl/libGLES_mali.so
I/OpenGLRenderer: Initialized EGL, version 1.4
D/OpenGLRenderer: Swap behavior 1
D/mali_winsys: EGLint new_window_surface(egl_winsys_display*, void*, EGLSurface, EGLConfig, egl_winsys_surface**, egl_color_buffer_format*, EGLBoolean) returns 0x3000,  [1440x2560]-format:1
V/InputMethodManager: Starting input: tba=android.view.inputmethod.EditorInfo@6534749 nm : org.tensorflow.yolo ic=null
I/InputMethodManager: [IMM] startInputInner - mService.startInputOrWindowGainedFocus
D/InputTransport: Input channel constructed: fd=83
I/Choreographer: Skipped 4146 frames!  The application may be doing too much work on its main thread.
I/CameraManagerGlobal: Connecting to camera service
I/TensorFlowInferenceInterface: Checking to see if TensorFlow native methods are already loaded
E/art: No implementation found for long org.tensorflow.contrib.android.RunStats.allocate() (tried Java_org_tensorflow_contrib_android_RunStats_allocate and Java_org_tensorflow_contrib_android_RunStats_allocate__)
I/TensorFlowInferenceInterface: TensorFlow native methods not found, attempting to load via tensorflow_inference
I/TensorFlowInferenceInterface: Successfully loaded TensorFlow native methods (RunStats error may be ignored)
D/AndroidRuntime: Shutting down VM
E/AndroidRuntime: FATAL EXCEPTION: main
Process: org.tensorflow.yolo, PID: 9387
java.lang.RuntimeException: Failed to load model from 'file:///android_asset/tiny-yolo-voc-graph.pb'
at org.tensorflow.contrib.android.TensorFlowInferenceInterface.(TensorFlowInferenceInterface.java:100)
at org.tensorflow.yolo.TensorFlowImageRecognizer.create(TensorFlowImageRecognizer.java:42)
at org.tensorflow.yolo.view.ClassifierActivity.onPreviewSizeChosen(ClassifierActivity.java:59)
at org.tensorflow.yolo.view.CameraActivity.lambda$setFragment$0$CameraActivity(CameraActivity.java:119)
at org.tensorflow.yolo.view.CameraActivity$$Lambda$0.onPreviewSizeChosen(Unknown Source)
at org.tensorflow.yolo.view.CameraConnectionFragment.setUpCameraOutputs(CameraConnectionFragment.java:291)
at org.tensorflow.yolo.view.CameraConnectionFragment.openCamera(CameraConnectionFragment.java:298)
at org.tensorflow.yolo.view.CameraConnectionFragment.access$000(CameraConnectionFragment.java:52)
at org.tensorflow.yolo.view.CameraConnectionFragment$1.onSurfaceTextureAvailable(CameraConnectionFragment.java:163)
at android.view.TextureView.getHardwareLayer(TextureView.java:389)
at android.view.TextureView.draw(TextureView.java:338)
at android.view.View.updateDisplayListIfDirty(View.java:17296)
at android.view.View.draw(View.java:18080)
at android.view.ViewGroup.drawChild(ViewGroup.java:3966)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3752)
at android.view.View.updateDisplayListIfDirty(View.java:17291)
at android.view.View.draw(View.java:18080)
at android.view.ViewGroup.drawChild(ViewGroup.java:3966)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3752)
at android.view.View.draw(View.java:18321)
at android.view.View.updateDisplayListIfDirty(View.java:17296)
at android.view.View.draw(View.java:18080)
at android.view.ViewGroup.drawChild(ViewGroup.java:3966)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3752)
at android.view.View.updateDisplayListIfDirty(View.java:17291)
at android.view.View.draw(View.java:18080)
at android.view.ViewGroup.drawChild(ViewGroup.java:3966)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3752)
at android.view.View.updateDisplayListIfDirty(View.java:17291)
at android.view.View.draw(View.java:18080)
at android.view.ViewGroup.drawChild(ViewGroup.java:3966)
at android.view.ViewGroup.dispatchDraw(ViewGroup.java:3752)
at android.view.View.draw(View.java:18321)
at com.android.internal.policy.DecorView.draw(DecorView.java:919)
at android.view.View.updateDisplayListIfDirty(View.java:17296)
at android.view.ThreadedRenderer.updateViewTreeDisplayList(ThreadedRenderer.java:692)
at android.view.ThreadedRenderer.updateRootDisplayList(ThreadedRenderer.java:698)
at android.view.ThreadedRenderer.draw(ThreadedRenderer.java:806)
at android.view.ViewRootImpl.draw(ViewRootImpl.java:3128)
at android.view.ViewRootImpl.performDraw(ViewRootImpl.java:2924)
at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:2516)
at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:1515)
at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:7091)
at android.view.Choreographer$CallbackRecord.run(Choreographer.java:927)
at android.view.Choreographer.doCallbacks(Choreographer.java:702)
at android.view.Choreographer.doFrame(Choreographer.java:638)
at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:913)
at android.os.Handler.handleCallback(Handler.java:751)
at android.os.Handler.dispatchMessage(Handler.java:95)
at android.os.Looper.loop(Looper.java:154)
at android.app.ActivityThread.main(ActivityThread.java:6682)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:1520)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1410)
E/AndroidRuntime: Caused by: java.io.IOException: Not a valid TensorFlow Graph serialization: NodeDef mentions attr 'dilations' not in Op<name=Conv2D; signature=input:T, filter:T -> output:T; attr=T:type,allowed=[DT_HALF, DT_FLOAT]; attr=strides:list(int); attr=use_cudnn_on_gpu:bool,default=true; attr=padding:string,allowed=["SAME", "VALID"]; attr=data_format:string,default="NHWC",allowed=["NHWC", "NCHW"]>; NodeDef: 0-convolutional = Conv2D[T=DT_FLOAT, data_format="NHWC", dilations=[1, 1, 1, 1], padding="VALID", strides=[1, 1, 1, 1], use_cudnn_on_gpu=true](Pad, 0-convolutional/filter). (Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).
at org.tensorflow.contrib.android.TensorFlowInferenceInterface.loadGraph(TensorFlowInferenceInterface.java:392)
at org.tensorflow.contrib.android.TensorFlowInferenceInterface.(TensorFlowInferenceInterface.java:96)
... 53 more

First, stop with Object.java String toString () and stW = Cannot find local variable 'stW'.
Best Regars,
keides2

Thanks for the details and the weight files, I will check it as soon as possible.

Hi,
I've checked your .pb file which you sent me in mail and surprisingly it worked for me. I've checked it on a Huawei P9 Lite and also on my laptop by using this implementation:  https://github.com/szaza/tensorflow-java-examples-spring. Here you can see the result:

Ok, it wasn't so accurate, probably it can be trained further.
I am wondering on this assertion error: AssertionError: expect 63102556 bytes, found 63471556. Have you changed the number of classes in the configuration file?

Thank you for checking the pb file. I was looking forward to hearing from you.
Classes and filters are changing. Could you please check the cfg file that I am sharing with you on google drive?
This device is Samsung. I will try another device.
If the number of learning times is small, will fatal error occur when reading .pb files?
Thank you again,
送信者: YOGA Tablet 2-830L
2018/07/09 4:11 Zoltán Szabó<EMAIL_ADDRESS>
Hi,
I've checked your .pb file which you sent me in mail and surprisingly it worked for me. I've checked it on a Huawei P9 Lite and also on my laptop by using this implementation: https://github.com/szaza/tensorflow-java-examples-spring. Here you can see the result:
Ok, it wasn't so accurate, probably it can be trained further.
I am wondering on this assertion error: AssertionError: expect 63102556 bytes, found 63471556. Have you changed the number of classes in the configuration file?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.
{"api_version":"1.0","publisher":{"api_key":"05dde50f1d1a384dd78767c55493e4bb","name":"GitHub"},"entity":{"external_key":"github/szaza/android-yolo-v2","title":"szaza/android-yolo-v2","subtitle":"GitHub repository","main_image_url":"https://assets-cdn.github.com/images/email/message_cards/header.png","avatar_image_url":"https://assets-cdn.github.com/images/email/message_cards/avatar.png","action":{"name":"Open in GitHub","url":"https://github.com/szaza/android-yolo-v2"}},"updates":{"snippets":[{"icon":"PERSON","message":"@szaza in #6: Hi,\r\nI've checked your .pb file which you sent me in mail and surprisingly it worked for me. I've checked it on a Huawei P9 Lite and also on my laptop by using this implementation: https://github.com/szaza/tensorflow-java-examples-spring. Here you can see the result:\r\n\r\nOk, it wasn't so accurate, probably it can be trained further.\r\n\r\nI am wondering on this assertion error: AssertionError: expect 63102556 bytes, found 63471556. Have you changed the number of classes in the configuration file?"}],"action":{"name":"View Issue","url":"https://github.com/szaza/android-yolo-v2/issues/6#issuecomment-403309516"}}} [ { "@context": "http://schema.org", "@type": "EmailMessage", "potentialAction": { "@type": "ViewAction", "target": "https://github.com/szaza/android-yolo-v2/issues/6#issuecomment-403309516", "url": "https://github.com/szaza/android-yolo-v2/issues/6#issuecomment-403309516", "name": "View Issue" }, "description": "View this Issue on GitHub", "publisher": { "@type": "Organization", "name": "GitHub", "url": "https://github.com" } }, { "@type": "MessageCard", "@context": "http://schema.org/extensions", "hideOriginalBody": "false", "originator": "AF6C5A86-E920-430C-9C59-A73278B5EFEB", "title": "Re: [szaza/android-yolo-v2] Conversion of .weights file to .pb file by darkflow (#6)", "sections": [ { "text": "", "activityTitle": "Zoltán Szabó", "activityImage": "https://assets-cdn.github.com/images/email/message_cards/avatar.png", "activitySubtitle": "@szaza", "facts": [ ] } ], "potentialAction": [ { "name": "Add a comment", "@type": "ActionCard", "inputs": [ { "isMultiLine": true, "@type": "TextInput", "id": "IssueComment", "isRequired": false } ], "actions": [ { "name": "Comment", "@type": "HttpPOST", "target": "https://api.github.com", "body": "{\n"commandName": "IssueComment",\n"repositoryFullName": "szaza/android-yolo-v2",\n"issueId": 6,\n"IssueComment": "{{IssueComment.value}}"\n}" } ] }, { "name": "Close issue", "@type": "HttpPOST", "target": "https://api.github.com", "body": "{\n"commandName": "IssueClose",\n"repositoryFullName": "szaza/android-yolo-v2",\n"issueId": 6\n}" }, { "targets": [ { "os": "default", "uri": "https://github.com/szaza/android-yolo-v2/issues/6#issuecomment-403309516" } ], "@type": "OpenUri", "name": "View on GitHub" }, { "name": "Unsubscribe", "@type": "HttpPOST", "target": "https://api.github.com", "body": "{\n"commandName": "MuteNotification",\n"threadId": 351979497\n}" } ], "themeColor": "26292E" } ]

Additional Information:
I also ran using locally installed tensorflow-java-examples-spring.
I replaced yolo-voc.pb in graph/YOLO and edited yolo-voc-labelas.txt.
The result is that accuracy is not good, but no error was issued.
Is it a device problem?
I can not think so because the original .pb file works without problems.

Hi,
I've found only the  .weights file, .pb file and DarknetConf files on the google drive?
Which link contains the .cfg file?
"The result is that accuracy is not good, but no error was issued." - it should be trained further, so the accuracy will increase.
I don't think so that it is a device problem, however both protobuff files ran on my phone without any problems.

I'm sorry I could not share the cfg file.
Please check the link below.
cfg
In addition, this phenomenon appears in Samsung SM - G 920 F (Android 7.0, API 24) and Xperia Z 5 (Android 7.0)

I noticed that this problem is the same as Isuue # 5.
Since the version of tensorflow-gpu is 1.8.0, should I modify the dependencies of build.gradle as follows?
build.gradle
dependencies {
compile 'org.tensorflow: tensorflow-android: 1.8.0'
}
The current dependencies are as below.
build.gradle
dependencies {
compile 'org.tensorflow: tensorflow-android: 1.2.0-preview'
}
Thank you in advance.

I'm glad that you finally managed to solve the problem and you did not give up to find the solution.
Thank you for your feedback, it is very useful.

It now works by modifying it as follows.
dependencies {
compile 'org.tensorflow: tensorflow-android:1.6.0'
}

Do not put a space between: and 1.
Tensorflow 1.8 seems to be still unstable.


PLW0120 (else clause on loop without a break statement) false positive inside a match

Excited by the new match support, I ran it on a big project which contains a lot of match statements. The following is the only issue.
Snippet:
while 1:
    match 10:
        case _:
            break
else:
    print('here')

Command: ruff --isolated --select PLW0120 x.py
Expected: no violations -- the else is definitely not redundant here.
Actual:
x.py:5:1: PLW0120 `else` clause on loop without a `break` statement; remove the `else` and de-indent all the code inside it

Version: 0.0.251

Thanks, will fix today.


Github action fails to install with error: repository not found

Expected Behavior
Install gh action successfully
Current Behavior
Fails
Steps to Reproduce
Add
name: Setup Teller
uses: spectralops/setup-teller@v1

or
name: Setup Teller
uses: spectralops/setup-teller@v2

in jobs.build.steps

@nikolasten the documentation is unfortunately wrong; please see my PR which covers the right action name.

Thanks! Closing this in favour of your issue and pr


Can't download thumbnail if conditionalcontainer is used

Problem occurs when Image field is inside the conditional container. Image is displaying correctly but when you'll try to hit download link then you'll download missing 'download.html' file instead of specific image.
Download is not working for fields inside the conditional container:

bump


.clang-tidy typo

.clang-tidy contains a typo which causes the readability suite of tests to remain disabled. Changing this line to read readability-* will enable these lints. A quick test with clang-tidy enabled emits a number of new warnings.
https://github.com/skypjack/entt/blob/a5ce90b268e2eac089b22b6b97ea97a8b15aedc1/.clang-tidy#L18
The lints include a number of warnings on items that may be deemed unnecessary such as requiring the integer suffixes to be uppercase, which may warrant being disabled as well.

🤦♂ thanks for pointing this out. 👍


Proxy does not call EndServiceCallback(11) after receiving OnHMIStatus(BACKGROUND).

Bug Report
Proxy does not call EndServiceCallback(11) after receiving OnHMIStatus(BACKGROUND).
Reproduction Steps

Register 2 apps:
・App1: navigation App for Android
・App2: navigation App for Android
Activate App1
App1 starts video streaming
Activate App2
App1 is not displayed in the HMI
It runs continuously in the background.

Expected Behavior

Activate App1: SDL->App1.OnHMIStatus(FULL)
App1 starts Video streaming
Activate App2: SDL->App2.OnHMIStatus(FULL)
Deactivate App1: SDL->App1.OnHMIStatus(BACKGROUND)
Proxy sends EndService request:SDL<-App1.EndServiceCallback(11)
But there is no implementation for step 5

Observed Behavior
Proxy does not call App1.EndServiceCallback(11) after receiving OnHMIStatus(BACKGROUND).
OS & Version Information

Android Version: 7.1
SDL Android Version: 4.9.0
Testing Against: SDL Core 5.1.3 + HMI running on our development environment.

Test Case, Sample Code, and / or Example App
App1:NaviVideoApp1_0.030000.zip
App2:NaviVideoApp2_0.030000.zip
Occurrence
100%
Recovery Method
None

Hi @zhouxin627, this has already been addressed in 4.10.0 with #1161.


Should get_number_of_crits() return crits for all kinds of damage?

Or only for damage from regular tower attacks?
This is important for towers like Shaman which has this effect:

Every tower below 10 level in 250 range receives 1 experience every time it crits.

The script for Shaman uses get_number_of_crits(). If get_number_of_crits() returned crits for all kinds of damage, then the effect could become too powerful for towers or items which deal extra damage.
Also, what about splash damage? If a tower hits and crits 5 creeps with splash damage, does that translate to +1 exp or +5 exp?
Need to test in original game.
Current behavior is that get_number_of_crits() returns non-zero only for crits from regular tower attacks.

Tested how splash and bounce attacks get affected by Shaman tower.
Result: each crit from splash/bounce attack is considred an individual crit. So it is correct behavior that Shaman's aura is more powerful for towers with multi target attacks.

According to issue #208, "damage" events don't get emitted for spell damage so it only makes sense to test Shaman towers with towers which deal extra attack damage.

Tested this with Shaman + Militia Watchtower.
Stacked crit chance for Watchtower so it's 14%:

Found that shaman's ability works on crits from both main attacks and extra attacks.
Exp from main attack crit:

Exp from extra attack crit:

Note: exp amount is 0.7 because it's attackspeed adjusted.
Note: I know that the second screenshot is extra attack because there's no floating text for crit damage like for main attack. Only crits from main attack damage have floating text in original youtd.
Therefore, get_number_of_crits() should return non-zero for crits  of all damage types. Main tower attacks, crits from do_attack_damage() and do_spell_damage().


Add Retry link for failed layers

Adds a retry link to layers that failed.
To test, find or construct a layer that has failed. Click the retry button which should roll things back to the transfer stage for layers with copied images, and the validation stage for layers with only uploaded images.
Connects #220

I wish I had this feature now :cry: But it's only useful if you can retry regardless of thumbnails.

Looks solid.

Looks good to me as well.

@azavea-bot rebuild

I have one concern, which might not generally play out but we should test it... There is nothing stopping the user from clicking the retry link a few times in a row. If that happened fast enough we might end up in a state where we kick off a bunch of messages for the same job at once. We don't have a safeguard against this right now.

I don't think that could happen because the retry button only shows when the layer is failed, and when retry is clicked, the state of the front-end model is changed so that it isn't failing.

Got it.

+1. I have verified that this works.


Getting error ivy-done: Wrong type argument: char-or-string-p, nil

It asked for input for documentation then it throws this error:
ivy-done: Wrong type argument: char-or-string-p, nil

I was getting this error when I hadn't yet installed a docset.
Strangely, I had to try installing a docset twice to get it to work (the first command created the directory).


How to run the test

Hi Alex,
I really interested on your awesome abba app but unfortunately I dont know RoR or sinatra but I really wanna know how it work. May I know how to run the /test/start.html and /test/complete.html? I've setup everything and the admin is works, but when I access these url in browser
http://localhost:3000/start and http://localhost:3000/complete
I got  406, and broken image
when I remove get '/start', :provides => 'image/gif'
I got experiment required
Any idea how to test your app in my local?
Thanks

No problem I got it, sorry my mistake, thanks


Websocket Auth Support

The auth flow currently uses Auth0HTTPBearer, which depends on a Request instance (http), for Websockets, I think we need to use cookies to pass the token?

I don't think cookies would work. The problem is that Websockets lacks standardisation when it comes to authentication, so you could do it in many many ways according to the security requirements of the app and the frontend constraints.
If you want to use auth0 access tokens, you first have to agree between frontend and backend on a mechanism to pass the token to the websocket server, this could be done as a request header or as the first connect message for example. Then, the most straighforward way (but not necessarily the best) in the backend is to do user = auth.get_user(SecurityScopes(...), creds= HTTPAuthorizationCredentials(...)), while making sure your auth has auto_error=False and the user is checked before allowing or breaking the connection. I don't think the fastapi hierarchical dependency injection would work for websockets.
This messiness of websockets is why, if you do not need bidirectional connection and only want to push data from backend to frontend (like notifications), server sent events are easier and safer because the auth is already solved on http endpoints. Here is a SSE library for fastapi:  https://github.com/sysid/sse-starlette

Thanks for the quick response @dorinclisu, in this case, I need bidirectional communication, cannot run away from WebSockets unfortunately, will try your suggestion and will share the codebase here.


Document Dashboard speed test feature

Which Cloudflare product(s) does this pertain to?
Analytics
Subject Matter
Speed test isn't documented
Content Location
Probably to the Dashboard section equivalent called "Speed".
Additional information
Customers wonder which IP addresses (IPv4 and IPv6) as well as which HTTP User-Agent are used by the speed test feature.
Customers also would like to know if this speed test is part of "Known Bots" in the WAF Rules.

Hey @wolruf. This would be best handled with a Jira ticket on the PCX space. Could you open one? On the other hand, I know the team is working on restucture this entirely, so this might be on the backlog for a while. Thanks!

Filed PCX-4387


Batch normalization performs different in tf.kera.Model and tf.estimator.Estimator.

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): v2.0.0-rc2-26-g64c3d38 2.0.0
Python version: 3.5
CUDA/cuDNN version: 10.1
GPU model and memory: GeForce RTX 2080Ti 11GB

Describe the problem
Batch normalization performs different in tf.kera.Model and tf.estimator.Estimator.
When run the following code, it is easy to get the output among ①Model ②Estimator transformed by tf.keras.estimator.model_to_estimator ③ Hand-made Estimator(pre-trained model is transformed to ckpt for ensuring the same weights).
VGG16(No BN layers in VGG): Three methods performs the same, ranging from 0.0 ~ 9.249033.
ResNet50: ①0.0 ~ 11.896655 ②0.0 ~ 11.621841 ③0.0 ~ 11.621841
MobileNetV2(alpha=1.0): ①0.0 ~ 3.871163 ②0.0 ~ 0.7918996 ③0.0 ~ 0.7918996
Conv2D + BN(Default initialization): ① performs the same to ②
Conv2D + BN(Random initialization): ① performs differ from ②
The above result means that all Keras pre-trained model cannot performs right in estimator.
I do the same comparison in TF 1.14, and it results the same thing.
So, how can I train a model with BN by using estimator correctly?
Source code / logs
import numpy as np
import tensorflow as tf
import logging

LOGGER = logging.getLogger("tensorflow")

# Build model.
def build_model(name, random_bn_params=None):
    if name == "ResNet50":
        model = tf.keras.applications.ResNet50(
            input_shape=(224, 224, 3),
            include_top=False,
            pooling="avg")
    elif name == "MobileNetV2_1.0":
        model = tf.keras.applications.MobileNetV2(
            input_shape=(224, 224, 3),
            include_top=False,
            pooling="avg")
    elif name == "VGG16":
        model = tf.keras.applications.VGG16(
            input_shape=(224, 224, 3),
            include_top=False,
            pooling="avg")
    else:
        if random_bn_params:
            params = {"beta_initializer": tf.random_normal_initializer(),
                      "gamma_initializer": tf.random_normal_initializer(),
                      "moving_mean_initializer": tf.random_normal_initializer(),
                      "moving_variance_initializer": tf.random_normal_initializer()}
        else:
            params = dict()
        layer = tf.keras.layers.Conv2D(1, (3, 3), use_bias=False)
        layer_bn = tf.compat.v1.keras.layers.BatchNormalization(**params)
        inp = tf.keras.layers.Input((224, 224, 3))
        out = layer(inp)
        out = layer_bn(out)
        model = tf.keras.Model(inputs=inp,
                               outputs=out)
    return model

# Model params.
MODEL_NAME = "VGG161"  # ResNet50、VGG16、MobileNetV2_1.0 are available, pre-trained models are loaded. Name not in the above three will build a Conv2D + BN network.
RANDOM_BN_PARAMS = True  # Whether to initialize BN  by randomized params when using a customized Conv2D + BN network.
INPUT_TENSOR_FUNC = lambda: tf.ones((1, 224, 224, 3))  # Fixed model input.

# Use Keras model to predict.
model = build_model(MODEL_NAME, RANDOM_BN_PARAMS)
model.compile(optimizer = tf.keras.optimizers.RMSprop(lr=0, decay=1.0),
              loss='mean_squared_error')
model_output = model.predict(INPUT_TENSOR_FUNC())

# Use Estimator transformed from Keras model to predict.
estimator = tf.keras.estimator.model_to_estimator(model)
estimator_output = next(estimator.predict(INPUT_TENSOR_FUNC))

# Use a handmake model equivalent to Keras model to predict.
def _init_variables_from_checkpoint(checkpoint_path, model_dir):
    flags_checkpoint_path = checkpoint_path
    # Warn the user if a checkpoint exists in the model_dir. Then ignore.
    if tf.compat.v1.train.latest_checkpoint(model_dir):
        LOGGER.info(
            "Ignoring model_init_name because a checkpoint already exists in %s." % model_dir)
        return None
    if flags_checkpoint_path is "":
        return None

    # Gather all trainable variables to initialize.
    variables_to_init = tf.compat.v1.trainable_variables()

    variables_to_init_dict = {var.name.rsplit(":", 1)[0]: var for var in variables_to_init}

    if tf.compat.v1.gfile.IsDirectory(flags_checkpoint_path):
        checkpoint_path = tf.compat.v1.train.latest_checkpoint(flags_checkpoint_path)
    else:
        checkpoint_path = flags_checkpoint_path

    LOGGER.info("Fine-tuning from %s." % checkpoint_path)

    # Gather all available variables to initialize.
    available_var_map = _get_variables_available_in_checkpoint(variables_to_init_dict,
                                                               checkpoint_path)

    init_op = tf.compat.v1.train.init_from_checkpoint(checkpoint_path, available_var_map)
    LOGGER.info("%d/%d variables in checkpoint has been restored." % (len(available_var_map),
                                                                      len(variables_to_init)))

    return tf.compat.v1.train.Scaffold(init_op=init_op)

def _get_variables_available_in_checkpoint(variables,
                                           checkpoint_path,
                                           include_global_step=False):
    """Returns the subset of variables in the checkpoint.

    Inspects given checkpoint and returns the subset of variables that are
    available in it.

    Args:
        variables: A dictionary of variables to find in checkpoint.
        checkpoint_path: Path to the checkpoint to restore variables from.
        include_global_step: Whether to include `global_step` variable, if it
            exists. Default True.

    Returns:
        A dictionary of variables.

    Raises:
        ValueError: If `variables` is not a dict.
    """
    if not isinstance(variables, dict):
        raise ValueError("`variables` is expected to be a dict.")

    # Available variables
    ckpt_reader = tf.compat.v1.train.NewCheckpointReader(checkpoint_path)
    ckpt_vars_to_shape_map = ckpt_reader.get_variable_to_shape_map()
    if not include_global_step:
        ckpt_vars_to_shape_map.pop(tf.compat.v1.GraphKeys.GLOBAL_STEP, None)
    vars_in_ckpt = {}

    # for key in ckpt_vars_to_shape_map:
    #     LOGGER.info("Available variable name: %s", key)

    for variable_name, variable in sorted(variables.items()):
        if variable_name in ckpt_vars_to_shape_map:
            if ckpt_vars_to_shape_map[variable_name] == variable.shape.as_list():
                vars_in_ckpt[variable_name] = variable
            else:
                LOGGER.warning("Variable [%s] is available in checkpoint, but has an incompatible "
                               "shape with model variable. Checkpoint shape: [%s], model variable "
                               "shape: [%s]. This variable will not be initialized from the "
                               "checkpoint.",
                               variable_name,
                               ckpt_vars_to_shape_map[variable_name],
                               variable.shape.as_list())
        else:
            LOGGER.warning("Variable [%s] is not available in checkpoint", variable_name)
    return vars_in_ckpt

def model_fn(features, labels, mode):
    # Set Keras learning phase for alter BatchNorm and Dropout performance.
    tf.keras.backend.set_learning_phase(mode == tf.estimator.ModeKeys.TRAIN)

    m = build_model(MODEL_NAME, RANDOM_BN_PARAMS)
    predictions = m(features)

    scaffold = None
    if MODEL_NAME in {"ResNet50", "VGG16", "MobileNetV2_1.0"}:
        scaffold = _init_variables_from_checkpoint("w:/keras/%s.ckpt" % MODEL_NAME, ".")

    # Create estimator_spec for Estimator.
    estimator_spec = tf.estimator.EstimatorSpec(
        mode=mode,
        predictions=predictions,
        scaffold = scaffold
    )
    return estimator_spec
estimator_handmake = tf.estimator.Estimator(model_fn=model_fn)
estimator_handmake_output = next(estimator_handmake.predict(INPUT_TENSOR_FUNC))

# Compare the output range.
print(np.max(model_output), np.min(model_output))
print(np.max(estimator_output[list(estimator_output.keys())[0]]), np.min(estimator_output[list(estimator_output.keys())[0]]))
print(np.max(estimator_handmake_output), np.min(estimator_handmake_output))

@mysephi
i ran the code shared, please confirm the error faced is the same in the gist here

@mysephi
i ran the code shared, please confirm the error faced is the same in the gist here

Yes. it is.

@mysephi
i ran the code shared, please confirm the error faced is the same in the gist here

You can also change MODEL_NAME and RANDOM_BN_PARAMS for more error mentioned above.

I was able to see some values getting generated from the code provided by you in Tf Nightly 2.6, please find the gist here and confirm if your issue is resolved.


Remove Pydantic dependency

Using Pydantic here is pretty much unnecessary, and removing the dependency will let the latest versions of zarr_checksum be used by packages that are still on Pydantic v1.

ping @jjnesbitt


Feature request: Alignment of multiple sequences with tree & identity matrix

It would be useful if there was a function that automatically did pairwise alignment for all sequences in a list creating a new data structure that contained:
(1) an identity matrix (or distance)
(2) the multiple sequence alignment
(3) a newick tree
(4) UPGMA results
or something along those lines.  This may already exist but I haven't figured it out yet.  I've been exploring skbio for the past week or so and it is awesome.  I want to incorporate skbio into my everyday pipeline.
Is there a way to get UPGMA alignment trees directly from a list of DNA objects?
Here's some code I was messing around with below; the objects in sequences are DNA objects.
# Identity matrix
sequences = sequences[:4]

def getIdentity(sequences):
    """
    Input: List of DNA objects
    Output: pd.DataFrame of identity
    """
    Ar_identity = np.zeros((len(sequences), len(sequences)))

    labels = list()
    for i, dna_A in enumerate(sequences): # Sorry if dna_A is misleading.  It's not the enzyme
        id = dna_A.metadata["id"]
        if id not in labels:
            labels.append(id)
        for j, dna_B in enumerate(sequences):
            aln = skbio.alignment.local_pairwise_align_ssw(dna_A, dna_B)
            Ar_identity[i,j] = aln[1]
    return pd.DataFrame(Ar_identity, index=labels, columns=labels)
DF_identity = getIdentity(sequences)
DF_identity

Here's an example of the identity matrix output that I got from Clustal2.1 w/ the same seqs.  I'm not sure how to get the identity matrix from the scores.
#
#
#  Percent Identity  Matrix - created by Clustal2.1
#
#
#
#      1: s4          100.00   41.99   42.99   42.99
#      2: s2           41.99  100.00   73.69   73.79
#      3: s1           42.99   73.69  100.00  100.00
#      4: s3           42.99   73.79  100.00  100.00

@jolespin thanks for your interest in scikit-bio! There currently isn't a direct/easy way to compute these results but it should be possible to combine functionality in scikit-bio and scipy to accomplish what you need.
I want to note that the alignment API in scikit-bio is pretty limited and not very easy to work with. We have plans to rewrite the alignment API from the ground up to support different underlying aligners and to support things like multiple sequence alignment and reference-based alignment. For now, you might be better off using an external tool to perform multiple sequence alignment, and then load the resulting MSA file using scikit-bio and continue "downstream" steps such as tree-building.
I recommend checking out the scikit-bio cookbook. This recipe shows how to perform progressive multiple sequence alignment with scikit-bio, and shows how to build a distance matrix and UPGMA tree from the MSA (UPGMA is performed using scipy.cluster.hierarchy.average). This recipe shows how to build a distance matrix and neighbor-joining tree (skbio.tree.nj) from an MSA.

Hey @jairideout thanks for getting back to me.  Those recipes are extremely useful!  I was following one of the tutorials to read in a fasta file using skbio.io.read but it's only taking in the first record of 7 entries in the fasta file.
for dna in skbio.io.read("/Users/jespinoz/Google/16s_seqs/s3974_ssu.fna", format="fasta", into=skbio.DNA):
    sequences.append(dna)
len(sequences)
1492

Which is the length of the 1st fasta record.  Am I doing this incorrectly?

Hey @jairideout thanks for the recipes; they are super useful!  I had a quick question about one of the recipes.  I tried loading in a fasta file using skbio.io.read and if I don't use the into it works fine but if I use into=skbio.DNA then it only reads in the first record.  Am I doing this incorrectly?

If you specify into=skbio.DNA (or skbio.Sequence/skbio.RNA/skbio.Protein), only the first FASTA record will be read into the specified sequence type. If you wish to load a particular (single) record, pass seq_num (see docs here for details about this parameter).
If you want to get a generator yielding FASTA records as scikit-bio sequence objects, don't provide into:
generator = skbio.io.read('seqs.fasta', format='fasta', constructor=skbio.DNA)
for seq in generator:
    # seq is an `skbio.DNA` object

The generator reader API is memory efficient because only a single FASTA record is loaded into memory at one time. You can specify the type of sequence object via constructor.
If your FASTA file is an MSA, you may want to load it as an skbio.TabularMSA object, which provides a convenient API for manipulating MSAs in memory:
msa = TabularMSA.read('seqs.fasta', constructor=skbio.DNA)

For more details/examples of reading/writing files with scikit-bio, see this cookbook recipe.


[HUDI-6648] Support building table views from existing files

Change Logs
This change allows us to support creating a table from a set of initial files that were not created by Hudi. This will allow us to bootstrap tables from an existing set of files instead of requiring users to rewrite tables with filenames in a format that Hudi expects (outside of the bootstrap feature)
This feature only works if metadata table is enabled.
This PR also contains some minor changes around unnecessary copying of arrays, use of LinkedList, and over use of Option when dealing with collections to reduce memory overhead.
Impact
This introduces more complexity when parsing a file status into a HoodieBaseFile but allows for more flexibility when initializing a Hudi table and unlocks ability to do a zero-copy-bootstrap of an existing set of files.
Risk level (write none, low medium or high below)
low
Documentation Update
Describe any necessary documentation update if there is any new feature, config, or user-facing change

The config description must be updated if new configs are added or the default value of the configs are changed
Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
ticket number here and follow the instruction to make
changes to the website.

Contributor's checklist

[x] Read through contributor's guide
[x] Change Logs and Impact were stated clearly
[x] Adequate tests were added if applicable
[ ] CI passed

@hudi-bot run azure