Empty parents array

I'm having some troubles getting reverse relations working at all with backbone-associations. Currently I'm rolling my own reverse relations but this is getting tiresome & difficult to maintain.
In the documentation, there seems to be a 'parents' array that refers to the parent models. Is there something I'm doing wrong in this fiddle? If you check the console its outputting an empty array.

@jameslai I'm trying to use the default reverse relation setup, but I'm running into an issue when using a custom map function:
http://jsfiddle.net/fs8whj6o/7/
The first console.log call should output the parents array, right? Is this a bug? I'm looking into the backbone-association code itself and if I figure out the problem I'll post a PR, but first just want to confirm that this isn't the intended behavior.

I'm not sure if the means you were adding locations was intentional, but I wouldn't advise adding items in such a way. That said, the content of your map wasn't valid without locations being present any longer. Could you clarify your intent with that method?
http://jsfiddle.net/fs8whj6o/8/

@jameslai I'm not sure what the question is exactly, but my use of the map function in my app is to fetch relations using IDs since the parent/child collections are fetched asynchronously and not made aware of each other during initial load of the page. I'm not sure what this has to do with the example, though I'm not too familiar with how backbone-association works internally, since the locations should be allocated when the page is loaded.
Maybe it would help to clarify my specific needs better... in my custom app I have relations setup like this:
Grandfather <-> Parent <-> Child
When a (grand)parent is clicked, the corresponding children in the heirarchy would be filtered. So if Grandfather 1 was clicked, Parent 1 & Children are filtered. Like so:
Grandfather1        Parent1         Child(1, 2, 3, 4)
Grandfather2        Parent2         Child(5, 6, 7)
Same thing if a parent was clicked:
Grandfather1        Parent1         Child(1, 2, 3, 4)
Grandfather2        Parent2         Child(5, 6, 7)
Another requirement, is that the parent heirarchy can be modified by children.
Currently I've removed backbone-association temporarily and am just using custom map functions internally, but would like to have proper relations setup in backbone. I've yet to find a backbone solution that works so easily and am confident backbone-association is a good fit but it might need some work with reverse relations first.
Or I could just be setting this up all wrong. If you have an easier way this could be done I'm all ears.
Cheers


⚠ Mobitopup has degraded performance

In 25ab835, Mobitopup (https://napcard.voz.tech) experienced degraded performance:

HTTP code: 200
Response time: 8415 ms

Resolved: Mobitopup performance has improved in 37c4e97.


feat(gdrive): remove google drive single user svg

Proposed Changes
Removing svg file and name of Google Drive Personal connector.
Potential Breaking Changes

Acceptance Criteria

[ ] The proposed changes are covered by unit tests
[ ] The potential breaking changes are clearly identified
[ ] README.md is adjusted to reflect the proposed changes (if relevant)

@GermainBergeron 👋, not sure if the svg is used somewhere else but I was recommended by @toofff  to see if we should removed it from plasma.
Please, let me know if it's not the case. I will close the PR.


278

Jira requests 278 and 279.
Changed Collaborative Exploration Initiative - Final report template must have "Report Period Start Date" field as mandatory.
279 - Added mandatory start date to commissioned industry study

@talebib ready for review


Add Support for Compact mode (js-Controller 2.0 Feature)

The upcoming version 2.0.0 of js-controller will support a new mode for low memory devices. For this, a few minor adjustments to the adapter must be made. Please follow these instructions: https://forum.iobroker.net/viewtopic.php?f=24&t=20387#p213466

This issue was moved to iobroker-community-adapters/ioBroker.wm-bus-community#6


[BUG] Python task Permission denied

Search before asking

[X] I had searched in the issues and found no similar issues.

What happened

python task Permission denied
What you expected to happen
怎么解决？
How to reproduce
k8s上部署后，创建python任务，执行报错Permission denied
Anything else
No response
Version
3.0.x
Are you willing to submit PR?

[X] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

还未修复？

I ran using k8s and also encountered this issue.

tenant_code is "default", and set to "root"
common.properties set sudo.enable=false

I have seen this simialr issue. How to fix this?


Fixed apiType and broken error handling

PR contains a couple of clean commits. Please look at each commit individually 👍
A couple of regression tests were broken, which made me curious.

API v2 pipeline changes the error handler, which broke the error handling for the site, because it expects an error and not an object with attrs. If you add a catch handler, you expect to get an error and not an object.

apiType was always undefined for functional calls e.g. api.v2.settings.edit({..} {context: {internal:true}}) returned true forisContentAPI(). The apiType was only handled on http layer, not on functional layer.

Please use "rebase & commit"!

[ ] review & test @allouis
[ ] review & test @gargol
[x] @gargol Kevin pushed some mobiledoc/html changes (https://github.com/TryGhost/Ghost/commit/4f9e687f62e0f1023c1d10aa6b06d82c523fc2a6), which breaks some regression importer & posts tests. Can you please ask him if they are okay to ignore for now?

This looks good - I will test A$AP

Have tested the part which had to do with changes in error handling, works great and is a lot cleaner than the previous implementation :+1:

@allouis Merged. Can you please confirm back if you have tested the apiType change? Thanks!


Added tests & removed unneeded

Codecov Report

Merging #164 into master will increase coverage by 0.75%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #164      +/-   ##
==========================================
+ Coverage   76.53%   77.29%   +0.75%
==========================================
  Files          40       40
  Lines        2779     2779
==========================================
+ Hits         2127     2148      +21
+ Misses        521      511      -10
+ Partials      131      120      -11

Impacted Files
Coverage Δ

vm/vmExpr.go
69.36% <0%> (+2.44%)
:arrow_up:

vm/vmStmt.go
72.89% <0%> (+2.49%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2f3628d...e826ba6. Read the comment docs.

@mattn
Any thoughts on this?

Opps. Sorry much. LGTM. Please fix conflicts.

I open https://github.com/mattn/anko/compare/pr164?expand=1

This is updated, no conflicts.

Thanks!


bower install 10.0.0-alpha8 creates weird bower path

I used bower i vaadin-core#10.0.0-alpha8 and now I have bower_components/vaadin-core-elements/vaadin-core.html but I expected to have bower_components/vaadin-core/vaadin-core.html

Never mind. User error.


Reject meshes with vertices whose unaggregated weights sum to more than 4

If a given vertex has large weight values for a large number (> 4) of bones, then the jcalcium compiler is going to pick the largest four and ignore the rest, even though by definition the contributions from the other bones are not insignificant. This could be surprising as the mesh deformations produced at run-time by jcalcium are clearly not going to match whatever came out of an authoring program such as Blender.
This sort of thing should be caught as early as possible, and so compilation should fail when this situation is detected.

Blender seems to be doing some sort of normalization internally. Need to find out exactly what!


Issuer claim (iss) Validation too generic

Environment

Spring boot starter:

active directory spring boot starter

OS Type: Linux

Java version:

1.9

Summary
I'm using azure spring boot package for validating the Bearer Token on the back-end, using client credentials flow for a system-to-system (StateLess); it works fine except for the issuer claim validation; the library accepts any token issued by either v1 or v2 endpoints but does not allows to restrics the 'iss' claim to a specific tenant.
So my backend application would accept a valid token from 'any' azure tenant, which is not the desired behaviour.
Reproduce steps
You need at least two tenants to test; get a token for each tenant and send a request to a back-end protected application
Expected Results
I'm excepting that the library accepts only token issued by my tenant.
Actual Results
Access to the back-end rest endpoint is allowed for both tokens

Sorry for the late response because of the Chinese New Year Public Holiday.  Your proposal is reasonable, SDK only validates the prefix of the "iss" claim and doesn't validate the tenant id. But according to our backlogs, we don't have enough resources to do it right now. There is a workaround that you can override UserPrincipalManager to implement your own validation logic. By the way,  the logic will be very easy to implement in class UserPrincipalManager, so PR is welcome. Thanks.

Is there any update on this request?  I would like this capability plus the ability to support multiple audiences.  This would be very easy if the DefaultJWTClaimsVerifier was an inject-able bean.
Per the comment above, I don't see how overriding UserPrincipleManager is easy.  If you give me a little more direction, I will definitely try.

I would appreciate any information you can provide.

@echurchm, sorry for the late response. Were are evaluating implementing this kind issuer check in the UserPrincipalManager but it will take a while. In the meantime, PR is welcome.


Switch ondemand_aws_cluster to ondemand_aws_docker_cluster.

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#1239  👈
#1235
#1238
#1237
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @rohinb2 and the rest of your teammates on  Graphite

Merge activity

Sep 11, 5:04 PM EDT: @rohinb2 started a stack merge that includes this pull request via Graphite.


Add water profile calculator

Hi, first of all, congratulations for your app man. I was looking for exactly this, a free recipe calculator qnd open source so i could adapt qnd contribute to the app

Maybe the app could have some calculations for the water profile? I think would be something very useful for brewers around. I will look unto your code man and try to come up with something to help you.


Can't set a variable value into bootstrap popover

('button').popover({
title: 'John',
content: 'Loading...'
});
var popover = $('button').data('popover');
setTimeout(function() {
var dt6 = new Date(stays[page-1].sty_DateCreated) ;
dt6.setDate(dt6.getDate() - 1) ;
$('button').attr('data-content',dt6);
var popover = $('button').data('popover');
popover.setContent();
popover.$tip.addClass(popover.options.placement);
},500);

Bootstrap 3 is no longer being officially developed or supported.
All work has moved onto our next major release, v4. As such, this issue or pull request is being closed as a "won't fix." For additional help and support, we recommend utilizing our community resources. Thanks for your understanding, and see you on the other side of v4!
<3,
@mdo and team


Module react/jsx-runtime not found

After I upgraded from 6.0.5 to 6.0.10 or 7.0.1 I get:
Failed to compile.

./node_modules/react-tiny-popover/dist/Popover.js
Module not found: Can't resolve 'react/jsx-runtime' in '/Users/.../node_modules/react-tiny-popover/dist'

I'm using React 16.8.6
Rollback!

i have the same question with<EMAIL_ADDRESS> did you find same solutions for this issue?

After I upgraded from 6.0.5 to 6.0.10 or 7.0.1 I get:
Failed to compile.

./node_modules/react-tiny-popover/dist/Popover.js
Module not found: Can't resolve 'react/jsx-runtime' in '/Users/.../node_modules/react-tiny-popover/dist'

I'm using React 16.8.6
Rollback!

i have the same question with<EMAIL_ADDRESS>did you find same solutions for this issue?

Any plans to fix this?

+1

add react-jsx-runtime  package, link is https://www.npmjs.com/package/react-jsx-runtime
add webpack alias, 'react/jsx-runtime': nodeModulesResolve('react-jsx-runtime'), like as follow example

@nuthinking @alex-mironov You should update your react version to 17. It helps me to use react-popover without an error "Can't resolve 'react/jsx-runtime'".


fix: #2773 - appropriate "clear?" and "delete?" messages for user lists

Impacted files:

app_en.arb: labels for user lists (clear? + delete?)
app_fr.arb: labels for user lists (clear? + delete?)
product_list_page.dart: now using the new "clear?" label for user lists
product_list_user_dialog_helper.dart: now localizing with the new "delete?" label for user lists

What

The "delete user list?" label was a hard-coded English string: now it's localized.
There was no "clear user list?" label: now there is.

Screenshot

case
screen

delete user list

clear user list

clear history

Fixes bug(s)

Fixes: #2773

Codecov Report

Merging #2778 (348f58e) into develop (2ea0da3) will decrease coverage by 1.73%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           develop   #2778      +/-   ##
==========================================
- Coverage     8.86%   7.12%   -1.74%
==========================================
  Files          161     218      +57
  Lines         6623   10650    +4027
==========================================
+ Hits           587     759     +172
- Misses        6036    9891    +3855

Impacted Files
Coverage Δ

...kages/smooth_app/lib/widgets/attribute_button.dart
0.00% <0.00%> (-92.00%)
:arrow_down:

...s/smooth_app/lib/data_models/user_preferences.dart
8.73% <0.00%> (-23.57%)
:arrow_down:

packages/smooth_app/lib/themes/smooth_theme.dart
62.26% <0.00%> (-20.72%)
:arrow_down:

...p/lib/generic_lib/dialogs/smooth_alert_dialog.dart
15.11% <0.00%> (-19.10%)
:arrow_down:

...mooth_app/lib/data_models/product_preferences.dart
21.68% <0.00%> (-9.75%)
:arrow_down:

packages/smooth_app/lib/main.dart
14.65% <0.00%> (-3.24%)
:arrow_down:

.../smooth_app/lib/pages/onboarding/welcome_page.dart
0.00% <0.00%> (-3.13%)
:arrow_down:

.../smooth_app/lib/pages/onboarding/scan_example.dart
0.00% <0.00%> (-2.28%)
:arrow_down:

...ackages/smooth_app/lib/pages/scan/scan_header.dart
2.50% <0.00%> (-2.27%)
:arrow_down:

...p/lib/pages/onboarding/consent_analytics_page.dart
0.00% <0.00%> (-1.57%)
:arrow_down:

... and 237 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

Thank you @VaiTon for your review!
I wouldn't vote against your idea of adding an "Are you sure?" title, but I'm not convinced enough to change my PR :)


add file attribute when load .py config file

Fix the exception when use __file__ attribute in .py config file.
eg:
import os
import dotenv

dotenv.load_dotenv(os.path.join(os.path.dirname(os.path.abspath(__file__)), '.env'))

databases = {
    'default': 'sql',
    'sql': {
        'driver': os.getenv('SQL_DRIVER'),
        'host': os.getenv('SQL_HOST'),
        'port': os.getenv('SQL_PORT'),
        'user': os.getenv('SQL_USER'),
        'password': os.getenv('SQL_PASSWORD'),
        'database': os.getenv('SQL_DATABASE')
    }
}

@luoxiaohei Can you add a test case for _get_config function that you've fixed ?
I'm considering this PR for MasoniteFramework/orator fork

@mandarvaze no problem!

@mandarvaze Hi, I have added test cases, but CI seems to have some problems. I don't think my code is causing CI to fail.

I don't think my code is causing CI to fail.

It is failing because some of the existing files aren't formatted "correctly", via black formatter.
Can you run pre-commit run --all-files locally ? It should "fix" the files, including two of your files. Then you need to git add -u and run pre-commit run --all-files  again. This time it should pass.
If you are having error running the command , look here
Finally, if you think it is too much, leave it. I can do it for you 😄

@mandarvaze Thank you! Now seems that everything is right.

Great. We are almost there. One last thing: Can you squash your 3 commits into a single commit, so that I can cherry pick the combined one.
Also, is this PR related to any open issue ? If yes, please mention.

@mandarvaze I have squashed commits. There is no open issue related to it.

@luoxiaohei Thanks for your contribution.
This PR is merged to MasoniteFramework/orator fork :
https://github.com/MasoniteFramework/orator/commit/d5b513c597ffc40ecff1e7f83680b7e50c3c5ace

@josephmancuso Hi, Can you merge this PR? I want to create a new PR to fix the error occurred by source code installation.

@luoxiaohei Feel free to create PR on MasoniteFramework/orator fork 😄

@mandarvaze OK! 😄


chore: bdd-spring-1580995074 to 0.0.1

chore: Promote bdd-spring-1580995074 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


plugin-vue doesn't respect configured loaders since 1.8

Describe the bug
plugin-vue merges inline TS scripts in SFC, and compiles them itself with ESBuild.
Doing so bypasses any loader configuration for TS, which is unexpected and creates problems (see below).
This is a regression that was introduced in 1.8 by this commit https://github.com/vitejs/vite/commit/93d9a2d175b1a1e3fe54197856a86887b1dadb74#diff-50999b66ad3fff48fd315417d4f8c98741da9f85a0fa6bb4ba212cc8f8bac967L271-R267
Specifically, the new behavior is to always merge inline ts scripts (previously it was done only for devServer):
https://github.com/vitejs/vite/blob/93d9a2d175b1a1e3fe54197856a86887b1dadb74/packages/plugin-vue/src/main.ts#L267-L268
Once merged, the inlined ts scripts are converted to JS by esbuild directly, without regard for any specific loader / config:
https://github.com/vitejs/vite/blob/93d9a2d175b1a1e3fe54197856a86887b1dadb74/packages/plugin-vue/src/main.ts#L190-L205
This breaks builds that have non-default configuration. Solving this will close the following issues:

#5044 (type generation with rollup-plugin-typescript2)
#5795 (const enums with rollup-plugin-typescript2)

The easiest fix is to go back to not inlining TS outside devServer.
A more complex fix is to apply loaders from config when building TS inside SFC. This is IMHO less preferred because it has a high impact on the performance of devServer.
Reproduction
See https://github.com/vitejs/vite/issues/5044 or https://github.com/vitejs/vite/issues/5795
System Info
System:
    OS: Windows 10 10.0.19042
    CPU: (12) x64 Intel(R) Core(TM) i7-9850H CPU @ 2.60GHz
    Memory: 7.69 GB / 15.79 GB
  Binaries:
    Node: 16.6.0 - C:\Program Files\nodejs\node.EXE
    Yarn: 1.22.10 - ~\AppData\Roaming\npm\yarn.CMD
    npm: 7.19.1 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: Spartan (44.19041.1266.0), Chromium (100.0.1185.50), ChromiumDev (102.0.1227.0)
    Internet Explorer: 11.0.19041.1566

Used Package Manager
npm
Logs
No response
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to https://github.com/vuejs/core instead.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

I made a PR for the simple fix (revert, don't inline during build): #7909


Broken heredoc parsing

The heredoc markers in the test data are not resolved to each other.
Test:
usage() {
    a=1
    cat <<EOF
Usage: rpmdev-newspec [option]... [appname]
EOF
}

I don't know if this is the same issue I have:
If I move the last EOD at the beginning of the line instead of intending it, then it displays correctly.


fix(errors): properly assign APIError.body

[x] I understand that this repository is auto-generated and my pull request may not be merged

Changes being requested
Simply assign the body from the parameter to the class variable.
Based on the comment in the code:
    ""The API response body.

    If the API responded with a valid JSON structure then this property will be the
    decoded result.

    If it isn't a valid JSON structure then this will be the raw response.

    If there was no response associated with this error then it will be `None`.
    """

Additional context & links
closes https://github.com/openai/openai-python/issues/948

Sorry about the bug and thanks very much for opening a PR!


Switch from file-based to DB cache

This change removes our production file-based default cache in favor of a default database cache.
There will be a corresponding PR to remove permission-setting on eregs_cache from our automations, but we already run createcachetable for our database post preview cache.
Checklist

[x] PR has an informative and human-readable title
[x] Changes are limited to a single goal (no scope creep)
[x] Code follows the standards laid out in the CFPB development guidelines

Can these references to eregs_cache go as well?

Is there any simple way to test this?

Should there be a TIMEOUT defined for the Database cache? It is currently set as None.
In cfgov/cfgov/settings/test.py it looks like cache is disabled for testing with a TIMEOUT of 0.
Also in cfgov/cfgov/settings/local.py cache is disabled when working locally.
When we get an export of production data, will there be cache defined in a database table?

Per the Django docs, the None will mean cache keys never expire by default.


accessing public files via read_file()

There is an issue with accessing public files using the read_file() function. It is probably because the function to work properly, it needs to know what type of file it attemts to read. However, at this moment it has access to the public link and not to the file extension.

I am not sure how to solve this issue. It works fine with the "dict" parameter, since then there is no  question about file extension (assuming it is a .json. So far the documentation uses this example and is not clear enough about that.


Added pip as a dependency in environment.yml

This PR adds pip as a dependency in environment.yml. This seems to be a push by conda as not having it included generates the following warning:
Warning: you have pip-installed dependencies in your environment file,
but you do not list pip itself as one of your conda dependencies.  Conda
may not use the correct pip to install your packages, and they may end
up in the wrong place.  Please add an explicit pip dependency.  I'm
adding one for you, but still nagging you.

@jdebacker

Codecov Report

Merging #505 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #505   +/-   ##
=======================================
  Coverage   67.43%   67.43%
=======================================
  Files          48       48
  Lines        5542     5542
=======================================
  Hits         3737     3737
  Misses       1805     1805

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5891dbc...12bc874. Read the comment docs.

@rickecon Thanks for this PR that addresses conda no longer including pip.


Installation error

Description
The installation of the new version for Pyfhel-CKKS (pip install Pyfhel==3.0.0b3) does not work.
Setup:

OS: Windows 10
Python: 3.9.8
C compiler version: MSC v.1929 64 bit (AMD64)
Pyfhel Version: 2.3.1

Error message

Weird! I cannot reproduce the error. In a fresh environment I get:

I'll have to try with Python 3.9 just to be sure. What is even weirder is that the version 3.0.0b3 is listed among the available versions. Can you check with pip install Pyfhel==3.0.0b2? Oh, and just in case, can you update pip to the latest version?

Gives the same error. Already with the latest version of pip

Is that the full error message? I have the feeling I'm missing something

This is the full error message

Collecting Pyfhel==3.0.0b2
Using cached Pyfhel-3.0.0b2.tar.gz (41.1 MB)
Installing build dependencies ... done
Getting requirements to build wheel ... error
ERROR: Command errored out with exit status 1:
command: 'C:\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\python.exe' 'C:\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pip_vendor\pep517\in_process_in_process.py' get_requires_for_build_wheel 'C:\AppData\Local\Temp\tmpogodqx4l'
cwd: C:\AppData\Local\Temp\pip-install-dndr5tvn\pyfhel_b66aaca0cd0544e582056915d00e9793
Complete output (53 lines):
Compiling C:\AppData\Local\Temp\pip-install-dndr5tvn\pyfhel_b66aaca0cd0544e582056915d00e9793\Pyfhel\Pyfhel.pyx because it depends on C:\AppData\Local\Temp\pip-build-env-468s3l0i\overlay\Lib\site-packages\Cython\Includes\libcpp\vector.pxd.
Compiling C:\AppData\Local\Temp\pip-install-dndr5tvn\pyfhel_b66aaca0cd0544e582056915d00e9793\Pyfhel\PyCtxt.pyx because it depends on C:\Users\Local\Temp\pip-build-env-468s3l0i\overlay\Lib\site-packages\Cython\Includes\libcpp\vector.pxd.
Compiling C:\AppData\Local\Temp\pip-install-dndr5tvn\pyfhel_b66aaca0cd0544e582056915d00e9793\Pyfhel\PyPtxt.pyx because it depends on C:\AppData\Local\Temp\pip-build-env-468s3l0i\overlay\Lib\site-packages\Cython\Includes\libcpp\vector.pxd.
Compiling C:\AppData\Local\Temp\pip-install-dndr5tvn\pyfhel_b66aaca0cd0544e582056915d00e9793\Pyfhel\PyPoly.pyx because it depends on C:\AppData\Local\Temp\pip-build-env-468s3l0i\overlay\Lib\site-packages\Cython\Includes\libcpp\vector.pxd.
[1/4] Cythonizing C:\AppData\Local\Temp\pip-install-dndr5tvn\pyfhel_b66aaca0cd0544e582056915d00e9793\Pyfhel\PyCtxt.pyx
[2/4] Cythonizing C:\AppData\Local\Temp\pip-install-dndr5tvn\pyfhel_b66aaca0cd0544e582056915d00e9793\Pyfhel\PyPoly.pyx
[3/4] Cythonizing C:\AppData\Local\Temp\pip-install-dndr5tvn\pyfhel_b66aaca0cd0544e582056915d00e9793\Pyfhel\PyPtxt.pyx
[4/4] Cythonizing C:\AppData\Local\Temp\pip-install-dndr5tvn\pyfhel_b66aaca0cd0544e582056915d00e9793\Pyfhel\Pyfhel.pyx
running egg_info
writing Pyfhel.egg-info\PKG-INFO
writing dependency_links to Pyfhel.egg-info\dependency_links.txt
writing requirements to Pyfhel.egg-info\requires.txt
writing top-level names to Pyfhel.egg-info\top_level.txt
cimporting numpy version '1.21.4'
reading manifest file 'Pyfhel.egg-info\SOURCES.txt'
Traceback (most recent call last):
File "C:\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pip_vendor\pep517\in_process_in_process.py", line 363, in
main()
File "C:\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pip_vendor\pep517\in_process_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "C:\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pip_vendor\pep517\in_process_in_process.py", line 13    File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2544.0_x64__qbz5n2kfra8p0\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2544.0_x64__qbz5n2kfra8p0\lib\distutils\dist.py", line 966, in run_commands
self.run_command(cmd)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2544.0_x64__qbz5n2kfra8p0\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "C:\AppData\Local\Temp\pip-build-env-468s3l0i\overlay\Lib\site-packages\setuptools\command\egg_info.py", line 299, in run
self.find_sources()
File "C:\AppData\Local\Temp\pip-build-env-468s3l0i\overlay\Lib\site-packages\setuptools\command\egg_info.py", line 306, in find_sources
mm.run()
File "C:\AppData\Local\Temp\pip-build-env-468s3l0i\overlay\Lib\site-packages\setuptools\command\egg_info.py", line 541, in run
self.add_defaults()
File "C:\AppData\Local\Temp\pip-build-env-468s3l0i\overlay\Lib\site-packages\setuptools\command\egg_info.py", line 585, in add_defaults
self.read_manifest()
File "C:\AppData\Local\Temp\pip-build-env-468s3l0i\overlay\Lib\site-packages\setuptools\command\sdist.py", line 195, in read_manifest
self.filelist.append(line)
File "C:\AppData\Local\Temp\pip-build-env-468s3l0i\overlay\Lib\site-packages\setuptools\command\egg_info.py", line 483, in append
path = convert_path(item)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2544.0_x64__qbz5n2kfra8p0\lib\distutils\util.py", line 123, in convert_path
raise ValueError("path '%s' cannot be absolute" % pathname)
ValueError: path '/make/rast/Documents/Pyfhel/Pyfhel/PyCtxt.cpp' cannot be absolute
WARNING: Discarding https://files.pythonhosted.org/packages/10/d9/118bf363f9cfcc6c5a3b6b17d6247420e95942df6dacd55ce81d5c358a3d/Pyfhel-3.0.0b2.tar.gz#sha256=4ec59ff4aa765545383e152b763d190a3f91f65f4c667d2ab7469147e03b8f3d (from https://pypi.org/simple/pyfhel/) (requires-python:>=3.7). Command errored out with exit status 1: 'C:\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\python.exe' 'C:\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\pip_vendor\pep517\in_process_in_process.py' get_requires_for_build_wheel 'C:\AppData\Local\Temp\tmpogodqx4l' Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement Pyfhel==3.0.0b2 (from versions: 2.0.0a2, 2.0.0a3, 2.0.0a4, 2.0.0a5, 2.0.1, 2.0.2rc1, 2.0.3, 2.0.4, 2.1.0, 2.2.0, 2.2.1, 2.2.3, 2.2.4, 2.2.5, 2.3.1, 3.0.0b1, 3.0.0b2, 3.0.0b3)
ERROR: No matching distribution found for Pyfhel==3.0.0b2

Thank you!

Same error with Python 3.10.1

Same error with Python 3.10.1

I will add a bit to it.
I tried to git clone (following the instructions of the repo) and then build it using Python3.10.1.
I got an issue while cloning:
remote: Enumerating objects: 358, done.
remote: Counting objects: 100% (27/27), done.
remote: Compressing objects: 100% (20/20), done.
remote: Total 358 (delta 7), reused 23 (delta 7), pack-reused 331
Receiving objects: 100% (358/358), 519.59 KiB | 4.64 MiB/s, done.
Resolving deltas: 100% (172/172), done.
Cloning into 'C:/Users/user/Pyfhel/Pyfhel/PALISADE'...
fatal: unable to access 'https://git.njit.edu/palisade/PALISADE/': Failed to connect to git.njit.edu port 443: Timed out

fatal: clone of 'https://git.njit.edu/palisade/PALISADE' into submodule path 'C:/Users/user/Pyfhel/Pyfhel/PALISADE' failed
Failed to clone 'Pyfhel/PALISADE' a second time, aborting

Executing the command python3 setup.py build returns, of course, an error.
So, I noticed that both the SEAL and HElib folders were empty. I manually cloned the HElib and the SEAL repositories into their respective directories, and tried to execute python3 setup.py build.
Now (I already had Microsoft Visual C++ >=14.0. If you don't have it installed, it will cause an error) it compiled the library successfully.
After that I also tried to import Pyfhel and everything seems to work.

ready had Microsoft Visual C++ >=14.0.

I have a similar issue while installing Pyfhel on Linux (i.e., fatal: unable to access 'https://git.njit.edu/palisade/PALISADE/' and SEAL and HElib folders were empty).

For the palisade REPO... you can try replacing the call to git.njit.edu
with the following
https://gitlab.com/palisade/palisade-release.git
I haven't figured out how to solve the missing SEAL part yet
On Thu, Jan 27, 2022 at 9:45 AM ShokofehVS @.***> wrote:

ready had Microsoft Visual C++ >=14.0.
I have a similar issue while installing Pyfhel on Linux (i.e., fatal:
unable to access 'https://git.njit.edu/palisade/PALISADE/' and SEAL and
HElib folders were empty).
—
Reply to this email directly, view it on GitHub
https://github.com/ibarrond/Pyfhel/issues/108#issuecomment-1023352516,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGAMLUSFETNUA5GIOM3OSD3UYFSATANCNFSM5IE7C36Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

I think Dr. Ibarrondo said you need to switch to the DEV branch... not the
main branch. That is where all of the fixes appear to be.
On Thu, Jan 27, 2022 at 11:55 PM Hassan Karim @.***>
wrote:

For the palisade REPO... you can try replacing the call to git.njit.edu
with the following
https://gitlab.com/palisade/palisade-release.git
I haven't figured out how to solve the missing SEAL part yet
On Thu, Jan 27, 2022 at 9:45 AM ShokofehVS @.***>
wrote:

ready had Microsoft Visual C++ >=14.0.
I have a similar issue while installing Pyfhel on Linux (i.e., fatal:
unable to access 'https://git.njit.edu/palisade/PALISADE/' and SEAL and
HElib folders were empty).
—
Reply to this email directly, view it on GitHub
https://github.com/ibarrond/Pyfhel/issues/108#issuecomment-1023352516,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGAMLUSFETNUA5GIOM3OSD3UYFSATANCNFSM5IE7C36Q
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Sincerely,
Hassan Karim
Thoughts in motion

I think Dr. Ibarrondo said you need to switch to the DEV branch... not the main branch. That is where all of the fixes appear to be. On Thu, Jan 27, 2022 at 11:55 PM Hassan Karim @.> wrote:
…
For the palisade REPO... you can try replacing the call to git.njit.edu with the following https://gitlab.com/palisade/palisade-release.git I haven't figured out how to solve the missing SEAL part yet On Thu, Jan 27, 2022 at 9:45 AM ShokofehVS @.> wrote: > ready had Microsoft Visual C++ >=14.0. > > I have a similar issue while installing Pyfhel on Linux (i.e., fatal: > unable to access 'https://git.njit.edu/palisade/PALISADE/' and SEAL and > HElib folders were empty). > > — > Reply to this email directly, view it on GitHub > <#108 (comment)>, > or unsubscribe > https://github.com/notifications/unsubscribe-auth/AGAMLUSFETNUA5GIOM3OSD3UYFSATANCNFSM5IE7C36Q > . > Triage notifications on the go with GitHub Mobile for iOS > https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 > or Android > https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. > > You are receiving this because you are subscribed to this thread.Message > ID: @.***> >
-- Sincerely, Hassan Karim Thoughts in motion

I tried to reach the DEV branch but my issue has remained on Windows 10, Python 3.10, Pyfhel 3.0.1 and MSVC 14.29.30133:
ERROR: Command errored out with exit status 1:
command: 'C:\Users\User\AppData\Local\Programs\Python\Python310\python.exe' 'C:\Users\User\AppData\Local\Programs\Python\Python310\lib\site-packages\pip_vendor\pep517\in_process_in_process.py' build_wheel 'C:\Users\User\AppData\Local\Temp\tmps5ci_qqn'
ERROR: Failed building wheel for Pyfhel
Failed to build Pyfhel
ERROR: Could not build wheels for Pyfhel, which is required to install pyproject.toml-based projects

Took quite a while, but hopefully Pyfhel==3.1.1 solves this! I've been coding and running the latest demos in Windows, so I have high hopes it will work!
Would it possibly work on branching and conditional statements as well?

@ShokofehVS You would need comparison for both branching and conditional statements, but it is not currently implemented in the Pyfhel backend. Your best choice is to use a different type of FHE scheme named TFHE. I suggest you to take a look at concrete

Yes exactly, I understand that TFHE works well on Comparison, however, I used Pyfhel entirely on my machine learning algorithms now only comparison statements are missing to be encrypted. Besides, there is no proper implementation of TFHE in Python. Would you mind suggesting one that can come along with Pyfhel?

Have a look at concrete-ml!

Closing, since the Pyfhel==3.1 version now installs correctly both in Windows and Linux!


Add WebSockets telemetry

Fixes #1209
I followed the approach outlined in #1209 of wrapping the upgrade feature and MITM the upgrade transport stream.
Public API is just the UseWebSocketsTelemetry extension method.
I fed the data to an EventSource event, but it could also alternatively be something like IWebSocketsTelemetryFeature.
I didn't know where to put the code so I made a WebSocketsTelemetry folder in the main project (all types are internal though).
ToDo: I will add tests when we're happy with the overall approach

Added more tests

Yes, I tried it with a simple demo app and the default Blazor template, and it's working as expected.


Affinity, Toleration and NodeSelector can be set

Changes on Pact Broker deployment:

enable setting NodeSelector, Toleration and Affinity;

@kbcasagrande Thanks for this. I'll give it a test and if all is good, we can merge this in.


chore(main): release 0.4.1

:robot: I have created a release beep boop
0.4.1 (2024-08-22)
Bug Fixes

updated sdk to 0.4.5 (736ff96)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

v0.4.1
:sunflower:


Device specific lag in ViewPager

Hi,
I'm experiencing very weird lag on my Moto X Pure.
I've SubsamplingScaleImageView (match parent,x,y) in a ViewPager without any fancy stuff, very much sticking to the sample, and somehow a lot of bigger Pictures (devices has 21mpx camera) are causing huge Lag (~50 frame jumps) on my Motorola X Pure.
The lag is, when swiping away from the already loaded picture.
First I thought, it was because of the size of the image. But other Images from the same camera/app are working well, even with bigger file sizes. I debugged a lot and tried a lot of stuff (i.e. setParallelLoadingEnabled, OffscreenPageLimit, ...) until I almost gave up and tried other devices. On the Oneplus One, and even on the MotoG 2014 the exact same pictures with the exact same AppBuild are not lagging and buttery smooth, which is pretty weird, as both devices have much weaker hardware. Therefore I assume there might be something in the operating system of my X, though I certainly have no clue where to point my finger at. The Oneplus runs Android 5, while the Moto G is on Android 6, like my X. So major Android version doesn't seem to be the factor as well. I attached two images from my Moto X camera. One with about 9 MB not causing lag, and the other with about 7 MB causing lag for the option to maybe reproduce the issue.
As it seems device/system related I thought it worth posting. It could hint towards iusses with specific library versions or similar...(?) Is there any method, which could lead to clues on the issues or is this maybe somthing known/expected?
Best regards
Simon
Edit: Sorry, filenames get changed with upload. The one with the car mirror is the bigger pic, that doesn't cause lag.

I'm also having lag in a viewpager and don't know the real reason. I'm using a very basic example, saving the images to the disk using Glide and then setting the ImageURI using the File that Glide gives me. So I'm actually using a file instead of bitmaps.
It only happens when I'm scrolling to the next page, scrolling to the previous page is very fluid. I'm also suffering from the visibility problem described in #127.
I'm testing with the images in this album: https://imgur.com/a/msKtB
They are big but not big enough to pass my device texture size (Nexus 5, 4096x4096). When I swipe to the next fragment of the viewpager, I see a lag spike in CPU usage, in the kernel. The image is not drawn at first, but even after being drawn the CPU spike continues. This happens with tiling on or off, doesn't matter.
Loading as a bitmap (instead of uri) solves the problem. However I lose the tiling.

Well it is the opposite for me. Scrolling away from the picture is causing lag, while scrolling on it, loading, zooming and so on are fine without any issues.

I have the same exact issue ya'll reporting. The only way I can make it stop is using a lower minimumTileDpi to like 104. The "un-zoomed" picture quality is degraded, but at least it doesn't lag anymore when swiping through the viewPager. I would love a public access to the reset() method so I could cancel every decoding work as soon the user swipes to another page.

Francis, is that specific for certain devices / images for you as well? If so, which devices and do you maybe have images that cause the issue?

Device is irrelevant. My test devices are the following: Nexus 6P, 6, 5X, 5, 4, 9, 10, One Plus One, One Plus 2 and One Plus X. Image type is irrelevant as well. Code is straightforward, pretty similar to the sample. My use case is simple, I use Glide for 80x80 thumbnails, I click a thumb, I call Glide to re-use the 80x80 cached bitmap and once that is up I pass the URI of the actual picture to this library and replace the Glide'd image. Sounds a bit overwhelming but it's way faster to re-use an already decoded, resampled & resized cached bitmap as a "preview" instead of relying on this lib to present a preview and then decode the picture. This is for my use case though, I spent a couple weeks debugging and testing all possible combinations. If I don't use a custom minimumTileDpi I encounter lag if I swipe on my ViewPager while the picture is being decoded by this lib, and if I swipe fast there are a couple hangs in what looks like the lib is still decoding the previous picture so it has to wait for something until it starts to decode the new picture in the foreground. And yes I use the parallel threading option.
My ViewPager is pretty lightweight too, I don't use fragments or anything, I simply replace the views inside so I save some overhead of re-creating new fragments for every swipe.
Nevertheless I love this lib, and I don't plan on not using it. I did a couple tests yesterday after reading this and I settled for 196 as minimumTileDpi. It's not as "fluid" as using lower dpi value (which is fairly obvious) the picture quality gain against my old value (104) is enough to justify and with tiling the users can just zoom for more detail.
I hope that was helpful.

Your input is appreciated. Though I don't believe we have the same problem. As clearly stated in the OP I'm experiencing issues only on one specific device with some images. Same Images, same code, even weaker device -> No issue.
To make sure I tried minimum DPI as well, which has no effect for me.

I Still wonder, why I'm facing it only on the one device/image combination.

I can almost guess it's because lower end devices have:

lower resolution so the inSample calculation is more aggressive on the decode
smaller max texture size which also limits the inSample calculation

On powerful devices we can assume there's enough juice, resolution & texture size to be a little more forgiven in the scaling math. This is really hard to get right, even Google on Photos hasn't nailed it and I can trigger lag fairly easily if I try enough.

After messing around I managed to improve my situation. Here's what I did to fix/diminish the lag:
I used this fork because it adds a setMaxBitmapDimensions method. In this method I used 1024 dpi as the maximum tile size.
Then I had to add previews. Before, I was using Glide to load/cache images. However in Glide you can't get the cached image file and a bitmap at the same time. This was fundamental. I had to change to UIL.
Now I tell UIL to load the image and to also generate a bitmap with the dimensions I provide. I get the cached file, use it as the base image and use the scaled bitmap as the preview image.
It's much more fluid now. Here's the code:
scaleImageView.setMaxBitmapDimensions(1024);

ImageSize imageSize = new ImageSize(scaleImageView.getWidth(), scaleImageView.getHeight());
imageLoader.loadImage(url, imageSize, null, new SimpleImageLoadingListener() {
    @Override
    public void onLoadingComplete(String imageUri, View view, Bitmap loadedImage) {
        File file = imageLoader.getDiskCache().get(url);

        BitmapFactory.Options options = new BitmapFactory.Options();
        options.inJustDecodeBounds = true;
        BitmapFactory.decodeFile(file.getAbsolutePath(), options);
        int height = options.outHeight;
        int width = options.outWidth;

        scaleImageView.setImage(ImageSource.uri(file.getAbsolutePath()).dimensions(width, height), ImageSource.bitmap(loadedImage));
    }
}, progressListener);

@kakai248 isn't that a bit overkill? You're basically doing this lib's job of decoding with another lib... The setMaxBitmapDimensions method seems useful, I'll play with it.

To use previews you have to pass a scaled image. If you pass the original image, it will lag. The only way I found was to download the original and also generate a scaled down bitmap.
Using preview makes the image show instantly, which makes the experience fast to the user. Not using it I could notice a small delay while the lib was loading the full image.

That makes sense!

Sorry I'm late to this party. Device specific issues are hard to reproduce but I think I can see why the Moto X Pure would have a problem - it has an insane screen density of 520ppi.
This library defaults to loading enough image data to at least match the resolution of the screen so with a screen that dense, and assuming a max bitmap size over 4096px, it will load almost any image you throw at it as one tile with a sample size of 1, which needs about 60Mb for your example images. Other devices with a smaller max bitmap size, a smaller screen, or a less dense screen will load the image in tiles and/or subsampled and perform better.
Setting minimumTileDpi to something more sane like 160 or 240 will reduce the quality but not noticeably for most people, and will usually reduce the amount of image data loaded to something the device can better cope with.
Re. a public reset(), use recycle() but this should only be called after the view is off screen. If you call it when the view pager transition starts the image will disappear.

Thanks @davemorrissey for that helpful information! That makes the issue perfectly clear and the solution very accepted.

my test devices log.
initialiseBaseLayer maxTileDimensions=16384x16384 getWidth()=720,sWidth()=1080,getHeight()=1280,sHeight()=10800 width scale=0.666667,height scale=0.118519 initialiseTileMap maxTileDimensions=16384x16384 TileLoadTask.doInBackground, tile.sRect=Rect(0, 0 - 1080, 10800), tile.sampleSize=2
maxTileDimensions is too large.. , i fix it use:
setMaxTileSize(4069, 4069);


VSCode will not open files with parentheses in the file path, MacOS

Does this issue occur when all extensions are disabled?: Yes/No

VS Code Version: 1.69.2
OS Version: MacOS 12.5

Version: 1.69.2 (Universal)
Commit: 3b889b090b5ad5793f524b5d1d39fda662b96a2a
Date: 2022-07-18T16:12:57.074Z (2 wks ago)
Electron: 18.3.5
Chromium: 100.0.4896.160
Node.js: 16.13.2
V8: <IP_ADDRESS>-electron.0
OS: Darwin arm64 21.6.0
Steps to Reproduce:

Create a file with parentheses in the folder name
mkdir ~/Downloads/test\ \(3\)
echo "hello" > ~/Downloads/test\ \(3\)/test.txt

Try to open the file
code ~/Downloads/test\ \(3\)/test.txt

You'll get a blank file in VSCode but the file opens elsewhere (nano/textedit/etc...). Rename the path without the parentheses, VSCode will open the file as expected.

note: It's not that uncommon to get a folder name with parentheses in it. Unzip something then unzip it again (as in you either forgot you'd already unzipped it or you downloaded a new version that has the same filename and so the folder already exists and MacOS added (2) to the unzip folder.


Add support for files larger that 32MB

It crashes when trying to download something bigger than 32MB.
I can upload some Luma3DS dumps if you need.

@TheSandBoxMKG Hi! Can you specify which from which type of source you're trying to download from (http, ftp, ...), and the full error message you get?

Yes, it's an HTTP source (An to be honest beforehand, it's one of those not so legal sites ^_^" but I don't think that's relevant to the issue in question)
Here you have the video some milliseconds before it crashes (it was under 32MB, I thought it would crash later but it's still around there), an image of the Luma3DS error screen and the dump Luma3DS did.
http://download4.erista.me/content/165?token=7187047ddc25bffb0b455ca3ac875d77

Wait, wrong project, I'm closing this issue. (There's another project also named MultiDL)
I'm sorry.


CUDA oom with batch size 1

I ran this command (with NVIDIA Apex installed):
tape-train-distributed transformer masked_language_modeling --nproc_per_node 4 --batch_size 1024 --grad
ient_accumulation_steps 256

and got the following output:
RuntimeError: CUDA out of memory. Tried to allocate 18.00 MiB (GPU 2; 7.43 GiB total capacity; 6.81 GiB already allocated; 6.94 MiB free; 6.94 GiB r
eserved in total by PyTorch)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/tape/utils/distributed_utils.py", line 38, in _wrap
    fn(**kwargs)
  File "/opt/conda/lib/python3.7/site-packages/tape/main.py", line 191, in run_train
    training.run_train(**train_args)
  File "/opt/conda/lib/python3.7/site-packages/tape/training.py", line 536, in run_train
    break
  File "/opt/conda/lib/python3.7/site-packages/tape/utils/utils.py", line 244, in __exit__
    raise RuntimeError(message)
RuntimeError: CUDA out of memory. Reduce batch size or increase gradient_accumulation_steps to divide each batch over more forward passes.
        Hyperparameters:
                batch_size per backward-pass: 1024
                gradient_accumulation_steps: 256
                n_gpu: 4
                batch_size per (gpu * forward-pass): 1

I got the same output with tape-train as with tape-train-distributed. I'm running on a machine with 4 GPUs. How can I be running out of memory if the batch size is (per gpu, per forward pass) is 1?  What can I do to actually run the training code on my machine?

I'm not sure how the error you report is possible given the command you listed (batch size 1024, 4 nodes, and 512 gradient accumulation steps gives you a batch size of 0.5 per GPU).
Regardless, if you can't run a batch size of 1, here are my suggestions in order of difficulty:

Try running with the --fp16 flag to use half precision, which should halve your memory footprint.
Train a smaller model.
Filter out the longest sequences from the dataset.
Play around with gradient checkpointing inside the transformer encoder. Some of this code is actually already implemented but is not enabled. Anyway, this can also be useful for increasing the batch size.
Use a larger GPU :)

Hope that helps.

I meant to write gradient_accumulation_steps=256 (which is clearly printed in the output). I'll try these things and get back to you, thanks.

Hi again @rmrao:
I successfully ran this for a decent amount of time before running out of memory again:
20/05/15 11:44:10 - INFO - tape.training -   [Ep: 0.84][Iter: 26680][Time: 278.81s][Loss: 2.2635][Perplexity: 10.734][LR: 9.459e-05]
Traceback (most recent call last):
  File "/opt/conda/bin/tape-train-distributed", line 8, in <module>
    sys.exit(run_train_distributed())
  File "/opt/conda/lib/python3.7/site-packages/tape/main.py", line 252, in run_train_distributed
    args.node_rank, args.master_addr, args.master_port)
  File "/opt/conda/lib/python3.7/site-packages/tape/utils/distributed_utils.py", line 169, in launch_process_group
    while not process_context.join():
  File "/opt/conda/lib/python3.7/site-packages/tape/utils/distributed_utils.py", line 122, in join
    raise Exception(msg)
Exception:

-- Process 1 terminated with the following error:
Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/tape/training.py", line 508, in run_train
    viz, num_log_iter, gradient_accumulation_steps)
  File "/opt/conda/lib/python3.7/site-packages/tape/training.py", line 299, in run_train_epoch
    loss, metrics = runner.forward(batch)  # type: ignore
  File "/opt/conda/lib/python3.7/site-packages/tape/training.py", line 85, in forward
    outputs = self.model(**batch)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/apex/amp/_initialize.py", line 197, in new_fwd
    **applier(kwargs, input_caster))
  File "/opt/conda/lib/python3.7/site-packages/tape/models/modeling_bert.py", line 484, in forward
    outputs = self.bert(input_ids, input_mask=input_mask)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/tape/models/modeling_bert.py", line 449, in forward
    chunks=None)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/tape/models/modeling_bert.py", line 346, in forward
    layer_outputs = layer_module(hidden_states, attention_mask)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/tape/models/modeling_bert.py", line 285, in forward
    attention_outputs = self.attention(hidden_states, attention_mask)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/tape/models/modeling_bert.py", line 242, in forward
    self_outputs = self.self(input_tensor, attention_mask)
  File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python3.7/site-packages/tape/models/modeling_bert.py", line 180, in forward
    attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))
RuntimeError: CUDA out of memory. Tried to allocate 108.00 MiB (GPU 1; 7.43 GiB total capacity; 6.73 GiB already allocated; 84.94 MiB free; 6.86 GiB reserved in total by PyTorch)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/lib/python3.7/site-packages/tape/utils/distributed_utils.py", line 38, in _wrap
    fn(**kwargs)
  File "/opt/conda/lib/python3.7/site-packages/tape/main.py", line 191, in run_train
    training.run_train(**train_args)
  File "/opt/conda/lib/python3.7/site-packages/tape/training.py", line 536, in run_train
    break
  File "/opt/conda/lib/python3.7/site-packages/tape/utils/utils.py", line 244, in __exit__
    raise RuntimeError(message)
RuntimeError: CUDA out of memory. Reduce batch size or increase gradient_accumulation_steps to divide each batch over more forward passes.

        Hyperparameters:
                batch_size per backward-pass: 1024
                gradient_accumulation_steps: 256
                n_gpu: 4
                batch_size per (gpu * forward-pass): 1

Additionally, here's an image of the perplexity during training. The reported perplexity (as ECE) in the paper was 6.01. Should I expect my model to reach a perplexity of 6.01, or is some configuration different? Although the perplexity was still slightly decreasing when I received the out-of-memory error, its minimum was 10.7; it did not seem to be approaching 6.01.

It looks like you were using a learning rate of 1e-4. That’s set as default in TAPE because we use that for downstream tasks. For training the language model, we used a learning rate of 1e-3.
Additionally, we saw continuous decrease in loss over the course of all 10 epochs of training, so you should expect to see it continue to decrease.

On May 15, 2020, at 9:03 AM, Jack Morris<EMAIL_ADDRESS>wrote:

Hi again @rmrao:
I successfully ran this for a decent amount of time before running out of memory again:
20/05/15 11:44:10 - INFO - tape.training -   [Ep: 0.84][Iter: 26680][Time: 278.81s][Loss: 2.2635][Perplexity: 10.734][LR: 9.459e-05]
Traceback (most recent call last):
File "/opt/conda/bin/tape-train-distributed", line 8, in
sys.exit(run_train_distributed())
File "/opt/conda/lib/python3.7/site-packages/tape/main.py", line 252, in run_train_distributed
args.node_rank, args.master_addr, args.master_port)
File "/opt/conda/lib/python3.7/site-packages/tape/utils/distributed_utils.py", line 169, in launch_process_group
while not process_context.join():
File "/opt/conda/lib/python3.7/site-packages/tape/utils/distributed_utils.py", line 122, in join
raise Exception(msg)
Exception:
-- Process 1 terminated with the following error:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/tape/training.py", line 508, in run_train
viz, num_log_iter, gradient_accumulation_steps)
File "/opt/conda/lib/python3.7/site-packages/tape/training.py", line 299, in run_train_epoch
loss, metrics = runner.forward(batch)  # type: ignore
File "/opt/conda/lib/python3.7/site-packages/tape/training.py", line 85, in forward
outputs = self.model(**batch)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/apex/amp/_initialize.py", line 197, in new_fwd
**applier(kwargs, input_caster))
File "/opt/conda/lib/python3.7/site-packages/tape/models/modeling_bert.py", line 484, in forward
outputs = self.bert(input_ids, input_mask=input_mask)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tape/models/modeling_bert.py", line 449, in forward
chunks=None)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tape/models/modeling_bert.py", line 346, in forward
layer_outputs = layer_module(hidden_states, attention_mask)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tape/models/modeling_bert.py", line 285, in forward
attention_outputs = self.attention(hidden_states, attention_mask)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tape/models/modeling_bert.py", line 242, in forward
self_outputs = self.self(input_tensor, attention_mask)
File "/opt/conda/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tape/models/modeling_bert.py", line 180, in forward
attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))
RuntimeError: CUDA out of memory. Tried to allocate 108.00 MiB (GPU 1; 7.43 GiB total capacity; 6.73 GiB already allocated; 84.94 MiB free; 6.86 GiB reserved in total by PyTorch)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/tape/utils/distributed_utils.py", line 38, in _wrap
fn(**kwargs)
File "/opt/conda/lib/python3.7/site-packages/tape/main.py", line 191, in run_train
training.run_train(**train_args)
File "/opt/conda/lib/python3.7/site-packages/tape/training.py", line 536, in run_train
break
File "/opt/conda/lib/python3.7/site-packages/tape/utils/utils.py", line 244, in exit
raise RuntimeError(message)
RuntimeError: CUDA out of memory. Reduce batch size or increase gradient_accumulation_steps to divide each batch over more forward passes.
    Hyperparameters:
            batch_size per backward-pass: 1024
            gradient_accumulation_steps: 256
            n_gpu: 4
            batch_size per (gpu * forward-pass): 1

Additionally, here's an image of the perplexity during training. The reported perplexity (as ECE) in the paper was 6.01. Should I expect my model to reach a perplexity of 6.01, or is some configuration different? Although the perplexity was still slightly decreasing when I received the out-of-memory error, its minimum was 10.7; it did not seem to be approaching 6.01.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

Thanks! I'll re-run and let you know how it goes.


Added facility to apply a filter function to captured output.

Adds a facility to apply a filter function to captured output. This is useful in situations where your experiment code uses a console based progress bar or similar in the text UI, but you would rather not have the observer store the control characters like backspaces and linefeeds in the database. This can be achieved by setting the captured_out_filter on the run or experiment as follows:
from sacred.utils import apply_backspaces_and_linefeeds

ex = Experiment('progress')
ex.captured_out_filter = apply_backspaces_and_linefeeds

It would be possible to also implement this kind of thing by subclassing the observers and filtering the text there, or by changing the observers directly. However, in this case, the user would need to configure all observers. It would also make it harder to implement adding the observer via the command line interface like you can do now.

Coverage increased (+0.3%) to 90.997% when pulling 4f3cca7b40a3014ce073739f525010cc085fb2bb on kevinmcguinness:feature/captured-out-filter into 9446a767d6811ef0ea5c826c4ca8d3715c07f669 on IDSIA:master.

Hey Kevin,
what a nice PR. You even added an example. Thanks a lot!
I was thinking about adding this feature myself, but I hesitated because I was concerned with the speed of the filter. But by making it optional and configurable this is not much of a problem. And maybe we can someday add a C-extension to do it fast.
I'm merging it now, but it'll need some work for the new 0.7 release since I'm trying to switch to file-descriptor based capturing.

@Qwlouse: would the new capturing be able to catch printf() and/or cout << from C/C++ extensions?

For the 0.7 release I've tried to switch to a  file-descriptor based capturing that would also catch printf and such. Right now, it doesn't yet work smoothly on windows. But I intend on fixing that problem soon. So from the next release on, all outputs to stdout, even from C extensions should be captured, yes.

Any chance you have a feature branch lying around somewhere? I'm on Linux exclusively anyway, and have a lot of cout happening. I can work around that for now with the logging mechanism of our cluster here, but collecting everything in sacred would be much preferred :)


bluesky-cmds has fallen behind dependencies: pin versions?

e.g. bluesky_queueserver_api.REManagerAPI has changed its kwargs in the last two years, which breaks our current arguments
https://github.com/bluesky/bluesky-queueserver-api/blame/cf80c4c4d3383b76d805c090fb5d3bd41b07233f/bluesky_queueserver_api/zmq/aio.py#L16
This could probably be fixed pretty easily, but there is no telling what other issues remain...

I was not able to reproduce this issue.  Inspection of code shows that the change in REManager kwargs is already handled in our code.  I think the issue I had was an improper config file (note that zmqproxy needs a different address format than the other 3 ports.
Corrective actions:

[ ] REMOVE bluesky version pins
[ ] make sure config defaults still work


[tesseract[training-tools]] build failure

I tried to install tesseract training tools (tesseract[training-tools]) but the build failed.
Package: tesseract[core,training-tools]:x64-windows -> 5.2.0#2
Host Environment

Host: x64-windows
Compiler: MSVC 19.33.31630.0
vcpkg-tool version: 2022-09-01-dfb82802c8cc562ce3b665a904a65b22314de724
vcpkg-scripts version: 01b29f6d8 2022-09-16 (18 hours ago)

To Reproduce
vcpkg install tesseract[training-tools]:x64-windows
Failure logs
-- Using cached tesseract-ocr-tesseract-5ad5325a0aa8effc47ca033625b6a51682f82767.tar.gz.
-- Cleaning sources at T:/Dev/vcpkg/buildtrees/tesseract/src/1682f82767-2e636863e8.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source T:/Dev/vcpkg/downloads/tesseract-ocr-tesseract-5ad5325a0aa8effc47ca033625b6a51682f82767.tar.gz
-- Using source at T:/Dev/vcpkg/buildtrees/tesseract/src/1682f82767-2e636863e8.clean
-- Found external ninja('1.10.2').
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Fixing pkgconfig file: T:/Dev/vcpkg/packages/tesseract_x64-windows/lib/pkgconfig/tesseract.pc
-- Using cached msys-mingw-w64-i686-pkg-config-0.29.2-3-any.pkg.tar.zst.
-- Using cached msys-mingw-w64-i686-libwinpthread-git-<IP_ADDRESS>73.5be8fcd83-1-any.pkg.tar.zst.
-- Using msys root at T:/Dev/vcpkg/downloads/tools/msys2/9a1ec3f33446b195
-- Fixing pkgconfig file: T:/Dev/vcpkg/packages/tesseract_x64-windows/debug/lib/pkgconfig/tesseract.pc
CMake Error at scripts/cmake/vcpkg_copy_tools.cmake:36 (message):
  Couldn't find tool "ambiguous_words":

      "T:/Dev/vcpkg/packages/tesseract_x64-windows/bin/ambiguous_words.exe" does not exist
Call Stack (most recent call first):
  ports/tesseract/portfile.cmake:74 (vcpkg_copy_tools)
  scripts/ports.cmake:147 (include)

Additional context
tesseract.zip
Complete build logs are attached.

Could you please check if you have the following files:
T:/Dev/vcpkg/packages/tesseract_x64-windows/bin/ambiguous_words.exe
T:/Dev/vcpkg/buildtrees/tesseract/x64-windows-dbg/bin/ambiguous_words.exe

Thanks very much.
"ambiguous_words.exe" does not exist at both locations you mentioned above.

Sorry for delay, I'm not able to repro this issue, could you please provide your configure log.

Package: tesseract[core,training-tools]:x64-windows -> 5.2.0#3
Host Environment

Host: x64-windows
Compiler: MSVC 19.34.31933.0
vcpkg-tool version: 2022-11-10-5fdee72bc1fceca198fb1ab7589837206a8b81ba
vcpkg-scripts version: 6f7ffeb18 2022-11-11 (9 hours ago)

To Reproduce
vcpkg install tesseract[training-tools]:x64-windows
Failure logs
-- Downloading https://github.com/tesseract-ocr/tesseract/archive/5ad5325a0aa8effc47ca033625b6a51682f82767.tar.gz -> tesseract-ocr-tesseract-5ad5325a0aa8effc47ca033625b6a51682f82767.tar.gz...
-- Extracting source T:/Dev/test/vcpkg/downloads/tesseract-ocr-tesseract-5ad5325a0aa8effc47ca033625b6a51682f82767.tar.gz
-- Using source at T:/Dev/test/vcpkg/buildtrees/tesseract/src/1682f82767-2e636863e8.clean
-- Found external ninja('1.11.0').
-- Configuring x64-windows
-- Building x64-windows-dbg
-- Building x64-windows-rel
-- Fixing pkgconfig file: T:/Dev/test/vcpkg/packages/tesseract_x64-windows/lib/pkgconfig/tesseract.pc
-- Using cached msys-mingw-w64-i686-pkg-config-0.29.2-3-any.pkg.tar.zst.
-- Using cached msys-mingw-w64-i686-libwinpthread-git-<IP_ADDRESS>73.5be8fcd83-1-any.pkg.tar.zst.
-- Using msys root at T:/Dev/test/vcpkg/downloads/tools/msys2/9a1ec3f33446b195
-- Fixing pkgconfig file: T:/Dev/test/vcpkg/packages/tesseract_x64-windows/debug/lib/pkgconfig/tesseract.pc
CMake Error at scripts/cmake/vcpkg_copy_tools.cmake:36 (message):
  Couldn't find tool "ambiguous_words":

      "T:/Dev/test/vcpkg/packages/tesseract_x64-windows/bin/ambiguous_words.exe" does not exist
Call Stack (most recent call first):
  ports/tesseract/portfile.cmake:74 (vcpkg_copy_tools)
  scripts/ports.cmake:147 (include)

Additional context

Entire logs found at T:\Dev\test\vcpkg\buildtrees\tesseract are attached.
logs.zip

The same problem with VS 17.4.1.

install tesseract[training-tools]:x64-windows

Confirmed.

The same problem with VS 17.4.1.

Confirmed.

I pre-installed "vcpkg-pkgconfig-get-modules:x64-windows" and it succeeded.
Is this relevant?

Nice catch. I tried a couple of installations on different machines, and all succeeded.
The only puzzle I have now is the installation for x64-windows-static variant. It failed with the same message - "ambiguous_words.exe" is missing.  But that was weeks ago after read your message.  Today I tried to install again for  the static variant and got different failure message.

Thanks very much for the hard works.
Although x64-windows still requires pre-installing "vcpkg-pkgconfig-get-modules" as mentioned by c-toyama, both x64-windows and x64-windows-static variants build successfully now.


Can't run a task twice

Config:
shell: {
    ls: {
        options: {
            async: false,
            stdout: true
        },
        command: 'ls'
    }
}
grunt.registerTask('ls', ['shell:ls', 'shell:ls']);

Then run:
grunt ls

Result with errors:
Running "shell:ls" (shell) task
Gruntfile.js

Running "shell:ls" (shell) task
Fatal error: Process :ls already started.

I can't reproduce this on 0.3.8.  Can we close this out @freewind?

Ok, close it, I will report when I have the same problem again


#418 Fix autocomplete of version

Fixes #418

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.03%) to 60.3%

Files with Coverage Reduction
New Missed Lines
%

com/devonfw/tools/ide/completion/CompletionCandidateCollector.java
2
91.3%

Totals

Change from base Build<PHONE_NUMBER>:
-0.03%

Covered Lines:
4982

Relevant Lines:
7953

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
2 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.03%) to 60.3%

Files with Coverage Reduction
New Missed Lines
%

com/devonfw/tools/ide/completion/CompletionCandidateCollector.java
2
91.3%

Totals

Change from base Build<PHONE_NUMBER>:
-0.03%

Covered Lines:
4982

Relevant Lines:
7953

💛  - Coveralls


Search Bar Custom Position

I have seen couple of issues from the past to have searchPlaceholder. Is there a way to have the search bar positioned next to the action buttons? If this feature already exists, please point me in the right direction. I couldn't find a proper solution.
Expected Behavior
To be able to change the position of the search bar right next to the search button/icon
Current Behavior
The search bar opens to the left of the toolbar while the button is on the right

+1 would love to see this

+1 up


Issues with FlutterFragment / FlutterView

I am using flutter fragment like this
        flutterFragment = Flutter.createFragment("lookingfor")
        val tx = supportFragmentManager.beginTransaction()
        tx.replace(R.id.container, flutterFragment)
        tx.commit()

but when i tap on textfield on my flutter screen the keyboard hide textfield and it doesn't scroll but when i use the same screen with flutter only project it work fine
 [✓] Flutter (Channel master, v0.8.5-pre.5, on Mac OS X 10.13.5 17F77, locale en-US)
[✓] Android toolchain - develop for Android devices (Android SDK 28.0.2)
[!] iOS toolchain - develop for iOS devices
    ✗ Xcode installation is incomplete; a full installation is necessary for iOS development.
      Download at: https://developer.apple.com/xcode/download/
      Or install Xcode via the App Store.
      Once installed, run:
        sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer
[✓] Android Studio (version 3.1)
[✓] IntelliJ IDEA Community Edition (version 2018.2.3)
[✓] VS Code (version 1.27.2)

Are you still seeing this with recent Flutter versions?

I can confirm the issue is still present in 1.7.8-hotfix3.
The resize of the view when keyboard opens doesn't work when embedded in a FlutterFragment. It works in a Flutter stand alone app, it works embedded in an iOS FlutterEngine.
I also tried to listen manually to a resize of the window with WidgetsBindingObserver but again, didChangeMetrics is only triggered in standalone or in iOS embed.
[✓] Flutter (Channel stable, v1.7.8+hotfix.3, on Mac OS X 10.14.5 18F132, locale fr-FR)
[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
[✓] Xcode - develop for iOS and macOS (Xcode 10.2.1)
[✓] iOS tools - develop for iOS devices
[✓] Android Studio (version 3.4)
[✓] VS Code (version 1.36.1)
[✓] Connected device (2 available)

The keyboard issue still persists. I used the FlutterFragment described here : https://flutter.dev/docs/development/add-to-app/android/add-flutter-fragment , but the keyboard hides my UI.
I created a simple demo project where the FlutterFragment is used so that you have an easier job seeing what I did: https://github.com/FrantisekGazo/flutter-fragment-keyboard-test
When I use the same Flutter UI but with regular FlutterActivity the UI adjusts when keyboard is shown. So it's definitelly an issue with the FlutterFragment.
Please have a look and advise what to do.
$ flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel unknown, v1.12.13+hotfix.7, on Mac OS X 10.15.2 19C57, locale en-GB)

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
[✓] Xcode - develop for iOS and macOS (Xcode 11.3.1)
[✓] Android Studio (version 3.5)
[✓] Connected device (2 available)

• No issues found!

@matthew-carroll  please reopen this issue cause I don't see an option for reopening it myself

@matthew-carroll ^


Introduce build.env for single-point of condifguration of environment variables

Describe what this PR does
The new build.env file contains environment variables that are used in several locations. By placing these in a single file, it will be easier to test with newer versions of tools and upgrade everything accordingly.
Is there anything that requires special attention
This PR is a work in progress. The DNM label has been set to indicate this.

[x] Golang version
[x] Golang CI version
[x] Gosec version
[x] Helm version
[x] minikube variables
[x] BASE_IMAGE for the FROM in container images
[x] CEPH_VERSION like "nautilus"
[x] e2e configuration
[x] build check for arm64/amd64 container images

Related issues
Fixes: #1180

@Madhu-1 @humblec PTAL, this is now ready for review.

@nixpanic typo here condifguration in the title of the PR.

overall it looks good to me. :+1:


Improve documentation with require elements for Create methods.

The create method documentation are listing the attributes that are required for the creation of rules, extension, etc...
However, it may be require to add additional information some times (relationship for ruleComponent) and it is shown in example but not really clear why it is there and so on...
Could you please update the require element section on top of the example ?
Thanks

@pitchmuc, if we take https://developer.adobelaunch.com/api/reference/1.0/rule_components/create/ as an example, are you wanting some basic description information that in addition to the attributes, you should also include relationships?  Or are you asking for some kind of expandable section (like the attributes) that describes relationships?

@thebenrob It is assumed that you know what relationships mean and how to structure it.
The approach that I follow and I recommend is to copy an existing structure but it doesn't exactly tell you what you are creating.
It would be great that we have more explanation on each type of information fill in the body.
An extended documentation on the attributes would be a great start.

We think this issue has been pretty well addressed with this new guide: https://developer.adobelaunch.com/api/guides/relationships/


How to define content-type?

I need to define Content-Type: multipart/form-data;  for the API endpoint in the generated doc.
How can I do that?
thank you.

Response or request?
On Mon, Feb 3, 2020, 14:44 Csaba Kissi<EMAIL_ADDRESS>wrote:

I need to define Content-Type: multipart/form-data; for the API
endpoint in the generated doc.
How can I do that?
thank you.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/mpociot/laravel-apidoc-generator/issues/684?email_source=notifications&email_token=ADNSD4JY5HOY7KMT6RBSPC3RBANSNA5CNFSM4KPFNKA2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IKTCYLA,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADNSD4NOKJJPDZ7X4KWDENDRBANSNANCNFSM4KPFNKAQ
.

@shalvah Request.

Have you checked the documentation? https://laravel-apidoc-generator.readthedocs.io/en/latest/config.html#headers


Whats the Algorithm to calculate the product dashboard

Whats the algorithm to calculate the average time for commit, build, environment on product dashboard.
I can see some problem with the current logic and would like to know the logic,

@shardulsrivastava: Please look at each of the files "*-data.js" under: "UI/src/components/widgets/product/".
The logic is using standard deviations and averages.
If you feel something is off in the logic, please feel free to submit a PR with justification on it.
Please let us know if you have any other questions. Thanks.

@shardulsrivastava: If you don't have any other questions at this time until later, I would like to go ahead and close this issue. Please let us know. Thanks !

@Sbrenthughes  the data m getting on the product dashboard for the environments, is not right, coz our builds automatically promote the code to the dev environments and then wait for the user input before promoting them further .

hence data in  COMMIT,BUILD,DEV should be in mins.
Any idea where can i look in the code for the issue.
As i can see that these details are being calculated from the timestamps with respect to the stages

these timestamps are being calculated for each commit after getting the deployable units for every environment of the component :
getEnvironments -> deployService.getDeployStatus -> getDeployStatus() -> Deployable Units are added from environment_component and environment_status tables.
Now assuming that there is only one deployment collector then there will be only one deployable unit for every environment of a component, that too will be the latest deployment i.e deployable unit.
now is it correct to calculate the stage timestamps based on the latest deployment , rather than calculating it based on the deployments for that commit.

My depth of knowledge about the specifics the time calculations is limited at best. I would assume that since the product dashboard is tracking commits as the come and go through the stages, it would be better to use the timestamps for the deployment of the commits.

@Sbrenthughes can you please point me to someone who has actually written this logic, i just want to know whats the reasoning behind calculating this way in backend.

Let me do some digging and I will followup

Hi Shardul,
Did you figure out why commits were in days?
What is your organization's pipeline structure?


Date time picker

Thankyou for this clean gui framework
I could not find a data time picker so that users can easily pick dates and time .

Just use something like https://flatpickr.js.org/.


0.6.0 - support for multiple stacked scenes

New features
Support for multiple stacked scenes
If there are multiple stacked views (dialogs, drawers, etc.) in a scene, the Screenplay will now
"remember" the state when you navigate to a different scene and restore the state correctly when you
navigate back.
API breaks

The Rigger class has been removed. It has been replaced with a boolean flag, Scene.isStacking.

superseded by https://github.com/weefbellington/screenplay/pull/68


Setting the http status code based on known error messages

Signed-off-by: Dani Louca<EMAIL_ADDRESS>Swarm uses the docker client API to communicate with the engine.
As you can see here and in case of an error, the serverResp.statusCode is not returned.
Currently swarm guesses the return code which makes it inconsistent with the status code documented here
This is not limited to ContainerCreate but because of the scope of the issue, the fix in this PR only addresses the container create api call only.
The error messages used in the helper function GetHTTPErrorStatusCode are from this engine method https://github.com/moby/moby/blob/17.05.x/api/server/httputils/errors.go#L56-L68

cc: @wsong

From the mentioned moby tree about this style of error checking:

  // FIXME: this is brittle and should not be necessary, but we still need to identify if
  // there are errors falling back into this logic.
  // If we need to differentiate between different possible error types,
  // we should create appropriate error types that implement the httpStatusError interface.

Note that we scrapped the httpStatusError mentioned there as well because it did not work well either (error types that encode an HTTP status code).

@cpuguy83
There is no question that the typed check error you mentioned is the correct approach;
but due to the current state of this project and the actual request (ie: just setting the correct http status code for CreateContainer)  I decided to take a simpler route and avoid bumping vendored dependencies and bringing in new packages.

In this case "simpler" is simply incorrect and brittle error handling... likely which doesn't even work if these are error messages from from docker.

@cpuguy83
This incorrectness is widespread across the swarm code base
@thaJeztah @wsong
If the goal if to revamp swarm error handling and then I will backup my changes and we can decide on how to proceed.

@wsong , @thaJeztah and @trapier had a discussion and we all agree that string error check is NOT the ideal fix and we agreed that the ultimate fix is to use type error check as suggested previously.
Due to the scope of changes, the error type check will not be implemented in this PR, instead, this PR will follow the same practice already in use across the swarm code base, and simply does a string match to when the image is missing and set the http code accordingly.
I updated the PR and simplified the change to a single error comparison.

@thaJeztah
For this particular change, the test you linked uses docker client which can't validate the fix and bubble up the http status code .
We'll need to expend the bats and helper functions to curl the endpoints and read the http status code.
@thaJeztah , let me know If an integration test is necessary , I can start looking into the scope of this change

I'm ok with merging as-is, but it would be good to have a follow up with a test, to prevent future regressions (this code/check being brittle as it is)


Add proposer field to PendingProposal.

This is slightly less space efficent than the way it was, but I feel we can sacrifice space efficency a little for safer typing.

Codecov Report
Base: 93.74% // Head: 93.73% // Decreases project coverage by -0.00% :warning:

Coverage data is based on head (2be295c) compared to base (fe0e182).
Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files
@@              Coverage Diff               @@
##           cwd-to-dao     #557      +/-   ##
==============================================
- Coverage       93.74%   93.73%   -0.01%
==============================================
  Files              42       42
  Lines            3819     3816       -3
==============================================
- Hits             3580     3577       -3
  Misses            239      239

Impacted Files
Coverage Δ

...opose/dao-pre-propose-approval-single/src/state.rs
100.00% <ø> (ø)

...se/dao-pre-propose-approval-single/src/contract.rs
87.40% <100.00%> (-0.28%)
:arrow_down:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Test failure: StartupTimeoutIsApplied

This issue was made automatically. If there is a problem contact ryanbrandenburg.
This test failed with the following error:
No entries matched by 'Application '/LM/W3SVC/1/ROOT' with physical root 'C:\\BuildAgent\\temp\\buildTmp\\912c20e9003548979073b124ddcb4988\\' failed to load clr and managed application. Managed server didn't initialize after 1000 ms.'
Expected: True
Actual:   False
   at Microsoft.AspNetCore.Server.IISIntegration.FunctionalTests.EventLogHelpers.AssertSingleEntry(String regexString, IEnumerable`1 entries) in /_/test/Common.FunctionalTests/Utilities/EventLogHelpers.cs:line 68
   at Microsoft.AspNetCore.Server.IISIntegration.FunctionalTests.EventLogHelpers.VerifyEventLogEvents(IISDeploymentResult deploymentResult, String[] expectedRegexMatchString) in /_/test/Common.FunctionalTests/Utilities/EventLogHelpers.cs:line 52
   at Microsoft.AspNetCore.Server.IISIntegration.FunctionalTests.StartupTests.StartupTimeoutIsApplied() in /_/test/Common.FunctionalTests/Inprocess/StartupTests.cs:line 261
--- End of stack trace from previous location where exception was thrown ---
------- Stdout: -------
| [0.002s] TestLifetime Information: Starting test StartupTimeoutIsApplied at 2018-11-06T10:46:25
| [0.020s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Information: Deploying [Variation] :: ServerType=IIS, Runtime=CoreClr, Arch=x64, BaseUrlHint=, Publish=True
| [0.031s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug: Processing C:\BuildAgent\temp\buildTmp\912c20e9003548979073b124ddcb4988\runtimes\unix\lib\netcoreapp2.0
| [0.031s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying System.Security.Principal.Windows.dll
| [0.033s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug: Processing C:\BuildAgent\temp\buildTmp\912c20e9003548979073b124ddcb4988\runtimes\unix\lib
| [0.034s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug: Processing C:\BuildAgent\temp\buildTmp\912c20e9003548979073b124ddcb4988\runtimes\unix
| [0.034s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug: Processing C:\BuildAgent\temp\buildTmp\912c20e9003548979073b124ddcb4988\runtimes\win\lib\netcoreapp2.0
| [0.034s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying System.Security.Principal.Windows.dll
| [0.036s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug: Processing C:\BuildAgent\temp\buildTmp\912c20e9003548979073b124ddcb4988\runtimes\win\lib
| [0.036s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug: Processing C:\BuildAgent\temp\buildTmp\912c20e9003548979073b124ddcb4988\runtimes\win
| [0.037s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug: Processing C:\BuildAgent\temp\buildTmp\912c20e9003548979073b124ddcb4988\runtimes
| [0.037s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug: Processing C:\BuildAgent\temp\buildTmp\912c20e9003548979073b124ddcb4988\wwwroot
| [0.037s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying static.txt
| [0.038s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug: Processing C:\BuildAgent\temp\buildTmp\912c20e9003548979073b124ddcb4988
| [0.038s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying aspnetcorev2_inprocess.dll
| [0.040s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying aspnetcorev2_inprocess.pdb
| [0.055s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying InProcessWebSite.deps.json
| [0.057s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying InProcessWebSite.dll
| [0.058s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying InProcessWebSite.exe
| [0.060s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying InProcessWebSite.pdb
| [0.061s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying InProcessWebSite.runtimeconfig.json
| [0.062s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.Authentication.Abstractions.dll
| [0.064s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.Authentication.Core.dll
| [0.065s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.Connections.Abstractions.dll
| [0.066s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.Hosting.Abstractions.dll
| [0.069s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.Hosting.dll
| [0.070s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.Hosting.Server.Abstractions.dll
| [0.071s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.Http.Abstractions.dll
| [0.072s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.Http.dll
| [0.077s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.Http.Extensions.dll
| [0.078s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.Http.Features.dll
| [0.079s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.ResponseCompression.dll
| [0.080s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.Server.IIS.dll
| [0.082s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.Server.IIS.pdb
| [0.083s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.Server.IIS.xml
| [0.084s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISDeployer Debug:   Copying Microsoft.AspNetCore.WebUtiliti
...

Other tests within that build may have failed with a similar message, but they are not listed here. Check the link above for more info.
This test failed on master.
CC muratg

This comment was made automatically. If there is a problem contact ryanbrandenburg.
Please use this workflow to address this flaky test issue, including checking applicable checkboxes and filling in the applicable "TODO" entries:

Is this actually a flaky test?

No, this is a regular test failure, fix the test/product (TODO: Link to commit/PR)
Yes, proceed below...

Is this test failure caused by product code flakiness? (Either this product, or another product this test depends on.)

[ ] File a bug against the product (TODO: Link to other bug)
Is it possible to change the test to avoid the flakiness?

Yes? Go to the "Change the test!" section.
No?

[ ] Disable the test (TODO: Link to PR/commit)
[ ] Wait for other bug to be resolved
[ ] Wait for us to get build that has the fix
[ ] Re-enable our test (TODO: Link to PR/commit)
[ ] Close this bug

Is it that the test itself is flaky? This includes external transient problems (e.g. remote server problems, file system race condition, etc.)

Is there is a way to change our test to avoid this flakiness?

Yes? Change the test!

[ ] Change the test to avoid the flakiness, for example by using a different test strategy, or by adding retries w/ timeouts (TODO: Link to PR/commit)
[ ] Run the test 100 times locally as a sanity check.
[ ] Close this bug

No?

Is there any logging or extra information that we could add to make this more diagnosable when it happens again?

Yes?

[ ] Add the logging (TODO: Link to PR/commit)

No?

[ ] Delete the test because flaky tests are not useful (TODO: Link to PR/commit)

This comment was made automatically. If there is a problem contact ryanbrandenburg.
There were 1 failures with about the same error on master at 1:31:38 AM:

StartupTimeoutIsApplied

This comment was made automatically. If there is a problem contact ryanbrandenburg.
There were 1 failures with about the same error on master at 1:15:09 PM:

StartupTimeoutIsApplied


AppEngine and google endpoints deployment to staging and production

breaking!! VERSION file defines the basepath of the API: /api/latest/public/utils/stats => /v3.0/platform/public/utils/stats if you have a strong opinion on what the URL should spell out instead of platform, now is the time
breaking /latest does not work anymore
we eat our own container-food: we deploy the nginx/uWSGI/supervisord container we have always used before
one google endpoints for each AppEngine version

http://branchname.projectid.appspot.com in dev
http://tagname.projectid.appspot.com in staging & prod

fix runscope API tests for all environments
deploy to opentargets-staging with a staging-* tag
deploy to opentargets-api-prod-us and opentargets-api-prod-eu with a prod-* tag
hardware resources, ES_URL, project IDs, etc are now all parameters to specify for each environment


UI: Ability to add drain timeout during service creation and upgrade

Rancher versions:
rancher/server: 1.6.11
Environment Template: (Cattle/Kubernetes/Swarm/Mesos) Cattle
With the 1.6.11, we will be supporting haproxy drain feature. To enable drain, parameter 'drain_timeout_ms' can be added in the service launch_config via rancher_compose.
Parameter details:
name: drainTimeoutMs (time in milliseconds to drain a service's containers)
[Field in rancher_compose is drain_timeout_ms]
Type: integer
Following are the UI changes needed:

We need the abillity to add this parameter while creating a Service via UI.
Also Upgrading a service should let user change the value of this parameter.
Note that this parameter can only be added for a Primary Service - so we don't need to provide this ability for sidekick services.

hi @prachidamle @vincent99 , where should we put the field in the UI when creating or updating a service?

It's both a property of the service resource and the upgrade action?  That's weird since none of the other upgrade options are like that (in 1.x).
For upgrade the obvious place is:

For create I guess underneath Command -> Auto Restart... there's really not a great place since none of the other options for stop signal, stop timeout, batch size, batch interval, start first, etc are part of the service definition in 1.x.

add this parameter while creating a Service

update this parameter while upgrading a Service

this parameter doesn't show in a Sidekick service

@vincent99 this parameter is part of launch_config of a service. So to add a value to it, it has to be on service creation page, since default is zero for this.
Also if user wants to change the value on a service, since the value belongs to service's launch_config, upgrade service is the action that will allow user to go change its value - I guess an edit service wont be applicable here?

@loganhz , @prachidamle , we can continue to have the "Darin Timeout" under command tab for service during upgrade workflow as well. Why should it be part of upgrade options ?

I misunderstood and thought there was an upgrade action field too; if it's just an editable part of the service then it can just be in the one place in @loganhz 's first screenshot for new or edit.

@vincent99 , there is no support for this field in standalone containers. Will introducing this field for service also result in this field being made available in standalone container create page ?

@sangeethah no, it will not show in standalone container page.

Available with v1.6.11-rc6

Tested with rancher-server version - v1.6.11-rc6
Able to set drain timeout during service creation and upgrade . This option is available under "command" tab.
This option is not presented for sidekicks and for standalone containers.