When summary has non numeric "pct", the output is "Unknown%"

When summary has "pct": "Unknown" the output is "Unknown%", should be 0%

fixed


Swiped cell state issue when table view is scrolling

In an early version if a cell is swiped, the table view cannot be scrolled.
But now table view can be scrolled if a cell is swiped. I don't know why we make this change.
In this case this swiped cell will back to normal state if user scrolls the table view, but didChangeSwipeState delegate is not called.
This issue can be reproduced in mail demo.
Thanks so much.

Fixed in 1.4.1 ;)


Build with the "org.embulk.embulk-plugins" Gradle plugin

Hi,
As we talked in the Developers' Meeting, we're going to use the new Gradle plugin to build Embulk plugins.
https://plugins.gradle.org/plugin/org.embulk.embulk-plugins
https://github.com/embulk/gradle-embulk-plugins
I'm trying to apply it with some upgrades. Can you have a look?

Once you run ./gradle gem, classpath/, lib/, and ?.gemspec are created under build/gemContents. I think it could be used for the same purpose.
We may have another separate task just to create this build/gemContents directory.
https://github.com/embulk/gradle-embulk-plugins/issues/70

Released with https://github.com/civitaspo/embulk-filter-expand_json/releases/tag/v0.4.0


Listen to Pico Button Presses

Hi everyone,
How does one listen for pico button presses using this library? I'd like to capture the event in my python code and then do something with it. I've searched high and low and can't quite figure this one out.

The buttons are available in bridge.buttons. Here's how Home Assistant subscribes to them: lutron_caseta/__init__.py:266-311

Thank you for the reply and forgive me for my ignorance. I'm newer to the programming world.
It looks to me like Home Assistant subscribes through this:
    for button_id in button_devices_by_id:         bridge_device.add_button_subscriber(             str(button_id),             lambda event_type, button_id=button_id: _async_button_event(                 button_id, event_type             ),         )
however, I don't get any responses that way. Here's my code:
`    def printThis(button_id, event_type):
print(button_id + event_type)
bridge.buttons
for b in buttons:
    print (b)
    bridge.add_button_subscriber(str(b), lambda event_type, button_id=b: printThis(button_id, event_type),)`

I do get the buttons initially, but then my process finishes without actually 'listening' for the button presses.
There is a section in the HA that shows:
    """Subscribe to lutron events."""     dev_reg = dr.async_get(hass)
however, as I dig deeper into that it doesn't appear to have much to do with lutron, though it may be more a fact that I'm just not understanding what's going on.
Your help is greatly appreciated!

Finally figured it  out.
You have to issue the following to get pylutron to "listen" for the button presses after you subscribe for them.
await bridge._monitor()

bridge is an instance of a Smartbridge connection:
    bridge = Smartbridge.create_tls(bridgeIp, "caseta.key", "caseta.crt", "caseta-bridge.crt")     await bridge.connect()

_monitor is a private method and you are not supposed to call it. Calling _monitor yourself after connect will most likely cause unexpected behavior.

You need to wait using asyncio (not the regular sleep function). The connection remains open in the background.


fix(pptx): accommodate invalid image/jpg MIME-type

As described in #3381, some clients, perhaps including Adobe PDF Converter, map JPEG images to the invalid image/jpg MIME-type. Prior to v1.0.0, python-pptx would not load these images, which caused image extraction to fail.
Update the python-pptx dependency to v1.0.0 or above to ensure this upstream fix is always available.
Fixes: #3381

Hi Matt, not sure what the process is for this these days. Should I run make pip-compile on this PR or just wait for this to get picked up by the next scheduled weekly run?

@scanny - You can run make pip-compile on this PR, and we can skip the dedicated dependency bump PR this week since this will cover it.


Update browser.js

What kind of change does this PR introduce?

Checklist:

[x] Unit Tests
[x] Documentation
[x] Update CHANGELOG.md
[x] Ready to be merged

What kind of change does this PR introduce?
Checklist:

[x]  Unit Tests
[x]  Documentation
[x]  Update CHANGELOG.md
[x]  Ready to be merged

demo/browser.js

🙂


Add sshttp Support

Nginx Proxy Manager is often used by professional home users who want to host several services by themselves. Those are also often confronted with another problem: workplace oder client networks which are "secured" by a proxy which doen't allow them to SSH to their beloved machines. SSHTTP uses several ways to distingish HTTP(S) from SSH traffic and re-route it accordingly.
Ideally you would define a seperate subdomain (ssh.example.com), set the scheme to ssh and define the host and port ssh shall be forwarded to. All irrelevant options (imho all other) could be disabled.

Perhaps even use SSLH since that also allows one to add OpenVPN to the list of things that can be directed through the SSL port.
I've been trying to do this with two separate dockers but it's not easy since the hosts get confused as to where the traffic originated.

I still think this would be a useful addition to Nginx Proxy Manager.


COMPAT: support sklearn 018

Initial attempt to support sklearn 0.18.

Coverage decreased (-0.4%) to 94.117% when pulling d5952966b006f55c012c42783a3a3eb1fdbb7a61 on sinhrks:sklearn_018 into 7b37ad4911eca6b6d9e7f29e5d175d74462c22f5 on pandas-ml:master.


chore: move style linters to the style namespace

[ ] depends on: #15970

@joneugster I have addressed your comment. Do you mind taking another look?

Looks good, but now you've created #16212 let's wait for that to be merged and resolve the merge conflicts here.

Let's land #15017 first; or I can split out another chunk of these changes.

Actually, #16265 already includes everything from this PR: let me close this one instead.


Server crashes when receiving little bit of load

Hello again, so basically like the title says, is only me using the server but it crashes and throws errors, is there any suggestion on how to tune it up to make it more performant and resilient?

What error do you see when it crashes? The first thing I would look at is to see if it's got enough memory. Otherwise, I'd suggest either trying to pre-generate what you expect to serve, or caching aggressively.

Hello, hope you guys had a great weekend.
This is the error we are getting, I'm using a t2.large on ec2 to host all of the service right now (pgsql and all dependencies)  going to work in a more robust environment with nginx as reverse proxy and caching results plus separate machines for the db and a fleet of 3 or 4 web servers. But my real question, do you have any specs recommended? and if I want it to put it under heavy loads any suggestion? Thanks for your help appreciate it.

We split our work up into several different service layers. The most important are:

Tileserver running on r3.xlarge instances. These only process "on-demand" tiles which are not part of the "Tiles of Interest" set, which (currently) lists all the tiles which have been requested and we have copies of on S3.
Tilequeue running on c4.xlarge instances. These process updates to the "Tiles of Interest", and so keeps S3 up-to-date so that those tiles can be served directly to clients using a service called Tapalcatl, which bypasses Tileserver and so helps reduce the number of requests.
PostgreSQL running on db.r3.2xlarge instances. These are in an RDS cluster, and so separate from the Tileserver and Tilequeue machines.

Looking at the monitoring for the Tileserver machines, it looks like they're hovering around 27GB heap usage. This seems like a lot, but it's hard to tell how much of that is actually in use without forcing more garbage collections to take place.

We run tileserver through gunicorn, and set worker_processes to 150 for the r3.xlarge instances. With a smaller instance size, the number of worker processes running should be tweaked accordingly.

Thank you very much for your response @rmarianski  @zerebubuth  I will replicate your settings and I will conduct some stress testing.
I'll close this issue and re-visit it later with some results from the tests.

Good luck!
We have a gitter channel, if you'd find it useful to chat.


Add YTMicro device support.

Signed-off-by: Major Lin<EMAIL_ADDRESS>

Thanks for retargeting to develop! 😄

Thanks for the updates! (And sorry, I'm way behind on issues!)

/azp run


Fix race-condition on memory pool

For https://github.com/cupy/cupy/issues/317#issuecomment-317614730

Remove in_use flag to avoid race-condition
Change free list from list to set so that we can find a free block fast without in_use flag
Add _free_lock and _in_use_lock to avoid race-condition with n_free_blocks(), used_bytes(), free_bytes(), and total_bytes().

Operations on set or dict are atomic in python, but locks are required for these functions which iterate set or dict.

I could not reproduce the error on my environment, so I could not write a test, but we've confirmed that the error was resolved on his environment.

jenkins, test this please

I benchmarked, and observed performance degradation.
import time
import cupy
pool = cupy.cuda.MemoryPool()

m = pool.malloc(2048)
del m

start_time = time.time()
for i in range(10000):
    m1 = pool.malloc(512)
    m2 = pool.malloc(512)
    m3 = pool.malloc(512)
    m4 = pool.malloc(512)
    del m2
    del m3
    del m1
    del m4
elapsed_time = time.time() - start_time
print(elapsed_time)

Before: 0.10920357704162598
After: 0.17422246932983398
I will try another version with no lock, but used_bytes(), etc is not thread-safe.

No lock version: 0.12729454040527344
https://github.com/cupy/cupy/compare/master...sonots:branch_fix_race_condition_memory_pool_no_lock?expand=1

I tried fastrlock https://github.com/scoder/fastrlock
https://github.com/cupy/cupy/commit/1a6fda6cfc4f3ae38cb246c10a64ff3bb2f67020
But, it did not improve performance at all ...
FastRLock version: 0.20251154899597168

As a general comment, it's often better (and always faster) to rely on the GIL, rather than using your own locking. But I admit that I haven't looked deeply enough into this to be sure that this can be done in the case at hand.

I tried to call fastrlock C API  > https://github.com/sonots/cupy/commit/5449a292e83f0728bad176c91e083d79170df16a
This version drastically improved performance!
FastRLock version: 0.11914539337158203

rebased to master

LGTM!


adding vector database and RAG integration

Added RAG functionality
Added vector store functionality

The Code Review now adds context  retrieved from a vector store containing data from a few coding standards PDFs

Summary by CodeRabbit

New Features

Introduced functionality to load, split, and upload PDF documents to Pinecone vector store using OpenAI embeddings.
Added a new environment variable PINECONE_API_KEY in the workflow configuration.

Improvements

Enhanced the call_openai function to handle context and improve prompt generation.
Updated dependencies for better performance and compatibility.
Modified the format_data_for_openai function to include system prompts.

Chores

Updated .gitignore to include standards and .env.
Added PINECONE_API_KEY to environment variables in the workflow configuration.

Bug Fixes

Corrected dependency versions and added missing libraries in requirements.txt.

Code Review
Summary
The pull request introduces several changes, primarily focused on integrating Pinecone for document storage and retrieval as part of the README update and code review automation process. The changes include updates to environment variables, new dependencies, new scripts, and modifications to existing functions to handle context retrieval using Pinecone.
Detailed Review
1. .github/workflows/update_readme.yaml

Change: Added PINECONE_API_KEY to environment variables.
Comment: This change is necessary for the integration with Pinecone, ensuring that the API key is securely passed to the workflow jobs.

2. .gitignore

Change: Added standards directory and .env file to the .gitignore.
Comment: This is a good practice to prevent sensitive information and unnecessary files from being tracked in the repository.

3. README.md

Change: Updated to reflect new functionalities and integrations.
Comment: The README now includes details about the Pinecone integration, which is essential for users to understand the new capabilities and how to utilize them.

4. constants.py

Change: Added EMBEDDING_MODEL and PINECONE_INDEX.
Comment: These constants are critical for the new embedding and vector store functionalities, ensuring consistent usage across the codebase.

5. main.py

Change: Modified the call_openai function to handle context retrieval using Pinecone and added a dummy commit to trigger the workflow.
Comment: The changes ensure that context is retrieved from Pinecone before generating the README update or code review comments, enhancing the accuracy and relevance of the outputs.

6. requirements.txt

Change: Added dependencies for Pinecone integration and environment variable management (langchain-pinecone, pinecone-client, pypdf, python-dotenv).
Comment: These dependencies are necessary for the new functionalities and ensure that all required packages are installed.

7. upload.py

Change: New script created to load PDF documents, split them into chunks, and upload them to Pinecone.
Comment: This script is essential for preprocessing documents and storing them in Pinecone, enabling efficient context retrieval during README updates and code reviews.

8. utility.py

Change: Modified call_openai to include context retrieval using Pinecone and updated format_data_for_openai to include the system prompt in the base prompt.
Comment: These modifications ensure that the context from stored documents is utilized, improving the quality of the generated content.

Commit Messages

The commit messages are descriptive and accurately reflect the changes made. This is important for maintaining a clear history of modifications and understanding the purpose of each change.

Recommendations

Code Quality: The code changes are well-structured and follow best practices. Ensure to handle exceptions and edge cases where Pinecone or OpenAI API calls might fail to enhance robustness.
Testing: Verify that the new Pinecone integration works as expected by adding unit tests or integration tests where applicable.
Documentation: Ensure that the README and other relevant documentation sections are kept up-to-date with any future changes to maintain clarity for users.

Conclusion
The code changes and commit messages are well-documented and align with the introduced functionalities. The updated README provides clear and comprehensive instructions, accurately reflecting the new features. The changes enhance the automation process, making it more robust and user-friendly, with clear separation of tasks for README updates, code reviews, and document management using Pinecone.
No further updates to the README are needed beyond those provided. The changes have maintained the existing style and clarity of the documentation.


Accept GitHub Sponsors

How about adding GitHub Sponsors to the list of support options?

Thanks for (ostensibly) your interest in supporting the project, but I'm going to close because the reasons articulated in https://github.com/badges/shields/issues/7415#issuecomment-1003780394 still hold true, including with GitHub Sponsors


Unclear how to apply new instance count to deployed group

I'm defining groups and sending the json to the group endpoint to create the group, that part works. If I go back and modify the instances to a new value (1->2) and then do a group update I get back a new deploymentID and version but I dont see it actually applying and new deployments/changes.
Is there documentation about group updates and is this expected to function?
aside: I'm using marathonctl

You can check the REST API docs for the PUT /v2/groups/{groupId} endpoint.
If you PUT a group and only change the instance count of an app, then it will be scaled, but the currently running tasks will not be restarted. If you modify any other app definition field, then all tasks will be restarted.
You can also scale up/down all the apps in a group by the same factor using the scaleBy attribute.
Examples:
POST /v2/groups:
{
  "id": "/prod",
  "apps": [
    {
      "id": "/prod/app1",
      "cmd": "echo `date`; sleep 5000",
      "instances": 4
    },
    {
      "id": "/prod/app2",
      "cmd": "echo `date`; sleep 5000",
      "instances": 4
    }
  ]
}

To scale /prod/app1 down to 2 instances and `/prod/app2' down to 3:
PUT /v2/groups/prod:
{
  "apps": [
    {
      "id": "/prod/app1",
      "instances": 2
    },
    {
      "id": "/prod/app2",
      "instances": 3
    }
  ]
}

Or to scale up both apps to 8:
PUT /v2/groups/prod:
{
  "scaleBy": 2
}


Support ddof parameter for std and var.

Supports ddof parameter for Frame.std and var.

Codecov Report

Merging #1986 (d26a5dd) into master (5c44ecc) will decrease coverage by 3.57%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master    #1986      +/-   ##
==========================================
- Coverage   94.54%   90.97%   -3.58%
==========================================
  Files          50       50
  Lines       10934    10835      -99
==========================================
- Hits        10338     9857     -481
- Misses        596      978     +382

Impacted Files
Coverage Δ

databricks/koalas/frame.py
93.67% <100.00%> (-3.07%)
:arrow_down:

databricks/koalas/generic.py
88.37% <100.00%> (-4.14%)
:arrow_down:

databricks/koalas/series.py
95.56% <100.00%> (-1.19%)
:arrow_down:

...bricks/koalas/tests/plot/test_frame_plot_plotly.py
26.43% <0.00%> (-73.57%)
:arrow_down:

databricks/koalas/plot/plotly.py
26.31% <0.00%> (-68.43%)
:arrow_down:

databricks/koalas/usage_logging/__init__.py
25.66% <0.00%> (-66.65%)
:arrow_down:

...ricks/koalas/tests/plot/test_series_plot_plotly.py
37.73% <0.00%> (-58.88%)
:arrow_down:

databricks/koalas/usage_logging/usage_logger.py
47.82% <0.00%> (-52.18%)
:arrow_down:

databricks/koalas/__init__.py
68.75% <0.00%> (-21.88%)
:arrow_down:

databricks/conftest.py
89.83% <0.00%> (-10.17%)
:arrow_down:

... and 23 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5c44ecc...d26a5dd. Read the comment docs.


Error in GeoDataRead

Hi,
I've just installed the package but ran into issues with GeoDataRead with the walkthrough example. Could you help to take a look?
Error in dplyr::summarize():
! Problem while computing Y = sum(Y).
Caused by error in sum():
! invalid 'type' (character) of argument
Run rlang::last_error() to see where the error occurred.
Attaching SessionInfo() output here, thank you!!
R version 4.2.2 (2022-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.utf8  LC_CTYPE=English_United States.utf8    LC_MONETARY=English_United States.utf8 LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8
attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base
other attached packages:
[1] GeoLift_2.6.01
loaded via a namespace (and not attached):
[1] augsynth_0.2.0         ggrepel_0.9.3          Rcpp_1.0.10            mvtnorm_1.1-3          lattice_0.20-45        tidyr_1.3.0            listenv_0.9.0
[8] zoo_1.8-11             assertthat_0.2.1       digest_0.6.31          foreach_1.5.2          utf8_1.2.2             parallelly_1.34.0      lfe_2.8-8
[15] R6_2.5.1               plyr_1.8.8             Boom_0.9.11            ggplot2_3.4.1          pillar_1.8.1           panelView_1.1.11       rlang_1.0.6
[22] rstudioapi_0.14        Matrix_1.5-1           stringr_1.5.0          BoomSpikeSlab_1.2.5    directlabels_2021.1.13 munsell_0.5.0          proxy_0.4-27
[29] compiler_4.2.2         pkgconfig_2.0.3        globals_0.16.2         CausalImpact_1.3.0     tidyselect_1.2.0       tibble_3.1.8           gridExtra_2.3
[36] quadprog_1.5-8         dtw_1.23-1             codetools_0.2-18       fansi_1.0.3            future_1.31.0          withr_2.5.0            dplyr_1.0.10
[43] MASS_7.3-58.1          grid_4.2.2             gsynth_1.2.1           xtable_1.8-4           gtable_0.3.1           lifecycle_1.0.3        DBI_1.1.3
[50] magrittr_2.0.3         scales_1.2.1           cli_3.4.1              stringi_1.7.12         reshape2_1.4.4         doRNG_1.8.6            doParallel_1.0.17
[57] xts_0.12.2             generics_0.1.3         vctrs_0.5.2            sandwich_3.0-2         Formula_1.2-4          iterators_1.0.14       tools_4.2.2
[64] glue_1.6.2             purrr_1.0.1            rngtools_1.5.2         MarketMatching_1.2.0   abind_1.4-5            parallel_4.2.2         colorspace_2.1-0
[71] LowRankQP_1.0.5        bsts_0.9.9

Hi @miaox018, thanks for sharing this issue with the community!
This issue is most often caused by non-numeric characters in the KPI variable (could be commas, currency signs, etc). I would recommend double-checking the format of your data with str(dataset). Nevertheless, we'll definitely add more user-friendly warnings to the GeoDataRead() function.
Please let us know if this works!

Hi again @miaox018! This is definitely an atypical behavior, as long as the KPI is numeric, the GeoDataRead() function should work. I would recommend trying  to re-format this variable (maybe with something like dataset$Y <- as.numeric(dataset$Y)). Alternatively, if you could share a reproducible example, we could dig deeper into this problem


No-op Component Parameter Renames

Currently F2 anywhere on a component start tag (ex. even component parameter) will trigger the rename window. We want to only show this window when a rename is triggered for the component name itself. This requires LSP platform to support textDocument/prepareRename. I've created a VSTS issue here. In the meantime, these changes ensure a rename operation on anywhere but the start of the component tag no-op.
Fixes: https://github.com/dotnet/aspnetcore/issues/26165
Issue for textDocument/prepareRename tracking: https://github.com/dotnet/aspnetcore/issues/26407

🆙 📅


chore: release master

:robot: I have created a release beep boop
configs: 1.32.1
1.32.1 (2023-06-03)
Bug Fixes

missing env variables (3c7198a)

configs-webpack: 1.15.1
1.15.1 (2023-06-03)
Bug Fixes

missing env variables (3c7198a)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/wholesome-ghoul/allaround-configs/releases/tag/configs-v1.32.1 :sunflower:

:robot: Release is at https://github.com/wholesome-ghoul/allaround-configs/releases/tag/configs-webpack-v1.15.1 :sunflower:


UI 3d assets not loading in

was playing project delta and wanted to check out my appearance but my avatar didnt load and the other items when i clicked inspect on them. it also happens in every other game that has 3d items in the UI

Do you have anti aliasing on?

Do you have anti aliasing on?

yea i had 8x msaa, i just tested it out with 4x msaa and now it seems to work

well a small thing to pay a big price for but i guess i gotta stick with the 4xmsaa now


Is there any way to integrate with fog?

Nice work on this, it's performing very well when testing so far.
One potential issue I'm encountering is that the AO effect is drawn over the top of any fog in the scene. I'm assuming because this effect is a screen space one it's not possible to actually draw it onto the material shaders before the fog. But I wondered if there was some depth information available and we could fade the opacity of the effect based on distance from the camera or something else that would give a reasonable approximation?
Is this kind of thing possible currently or any plans to implement?

Yes, I could modulate the effect by fog. This feature will be added in the next update.

Fog has been added!

Here's a picture of (overdone) bloom working with N8AOPass... try updating to 1.6.1.

Try - instead of this:
this.effectPass = new gD(this.camera,this.aaEffect,this.brightnessContrastEffect,this.bloomEffect),
This:
this.effectPass = new gD(this.camera,this.brightnessContrastEffect,this.bloomEffect, this.aaEffect),
As the AA effect should come after everything.
BTW this works with the latest version:
   const composer = new EffectComposer(renderer);
   composer.addPass(new RenderPass(scene, camera));
   const n8aopass = new N8AOPostPass(
       scene,
       camera,
       clientWidth,
       clientHeight
   );
   composer.addPass(n8aopass);
   composer.addPass(new EffectPass(camera, new BloomEffect({
       luminanceThreshold: 0.0,
       intensity: 5.0
   }), new SMAAEffect({
       preset: SMAAPreset.ULTRA
   })));
   ```

Thank you for that. I see what the difference is now.
I'm using a selective bloom technique where I set the bloom effect luminance threshold to 1 and then set the materials I want bloom on to be toneMapped: false and the emissiveIntensity to a high value like 8. Then only those objects will have the bloom effect applied. Could it be that because the N8aopass goes first it renders a tone mapped output or some other way of clipping the colours back into the normal range of 0 - 1?
If you try this code sandbox and comment out the aoPass on line 79 you will see the bloom effect is visible again.
https://codesandbox.io/s/sparkling-cdn-qslj6y?file=/src/index.js

N8AO uses non-float render targets... that could definitely be the problem. Next update (will be released tonight/early tmrw) will add support for floating render targets.

Should be fixed now.

Yep, it's working great. Thanks so much really appreciate the great work and fast updates!


Update README.md

Updating Custom Dockerfile and Community image with the last version of SonarQube and AEM.

Coverage remained the same at 94.665% when pulling 6fe07ffd6bc39142ce0192fa6d46e92d4b8000e2 on nomorehugs:patch-1 into cdce245dc5f932402f246241381e0a53df7c4390 on Cognifide:master.


RUMM-1001 Handle types with additionalProperties in JsonSchema

What does this PR do?
Handle the additionalProperties feature of JsonSchema (described here).
Motivation
Improve the way we handle the customTimings for view events (cf RUMF-783)

Codecov Report

Merging #473 (d42f533) into master (5ecdfeb) will decrease coverage by 0.13%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #473      +/-   ##
============================================
- Coverage     90.15%   90.02%   -0.13%
  Complexity     1359     1359
============================================
  Files           160      160
  Lines          4641     4641
  Branches        518      518
============================================
- Hits           4184     4178       -6
- Misses          286      291       +5
- Partials        171      172       +1

Impacted Files
Coverage Δ
Complexity Δ

...internal/tracking/JetpackViewAttributesProvider.kt
73.68% <0.00%> (-15.79%)
8.00% <0.00%> (-1.00%)

...nternal/tracking/OreoFragmentLifecycleCallbacks.kt
89.47% <0.00%> (-5.26%)
14.00% <0.00%> (-1.00%)

...oid/rum/tracking/NavigationViewTrackingStrategy.kt
86.27% <0.00%> (-1.96%)
14.00% <0.00%> (-1.00%)

...ernal/instrumentation/gestures/GesturesListener.kt
89.74% <0.00%> (-1.71%)
44.00% <0.00%> (-1.00%)

...tadog/android/log/internal/domain/LogSerializer.kt
90.00% <0.00%> (-1.43%)
28.00% <0.00%> (ø%)

...g/android/rum/internal/domain/scope/RumEventExt.kt
87.64% <0.00%> (-1.12%)
0.00% <0.00%> (ø%)

...l/net/info/BroadcastReceiverNetworkInfoProvider.kt
81.05% <0.00%> (ø)
28.00% <0.00%> (ø%)

...android/rum/internal/ndk/DatadogNdkCrashHandler.kt
82.95% <0.00%> (+0.78%)
25.00% <0.00%> (+1.00%)

...c/main/kotlin/com/datadog/android/DatadogConfig.kt
94.90% <0.00%> (+1.02%)
12.00% <0.00%> (+2.00%)

...oid/src/main/kotlin/com/datadog/android/Datadog.kt
89.41% <0.00%> (+1.18%)
29.00% <0.00%> (+1.00%)

Codecov Report

Merging #473 (d42f533) into master (5ecdfeb) will decrease coverage by 0.13%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #473      +/-   ##
============================================
- Coverage     90.15%   90.02%   -0.13%
  Complexity     1359     1359
============================================
  Files           160      160
  Lines          4641     4641
  Branches        518      518
============================================
- Hits           4184     4178       -6
- Misses          286      291       +5
- Partials        171      172       +1

Impacted Files
Coverage Δ
Complexity Δ

...internal/tracking/JetpackViewAttributesProvider.kt
73.68% <0.00%> (-15.79%)
8.00% <0.00%> (-1.00%)

...nternal/tracking/OreoFragmentLifecycleCallbacks.kt
89.47% <0.00%> (-5.26%)
14.00% <0.00%> (-1.00%)

...oid/rum/tracking/NavigationViewTrackingStrategy.kt
86.27% <0.00%> (-1.96%)
14.00% <0.00%> (-1.00%)

...ernal/instrumentation/gestures/GesturesListener.kt
89.74% <0.00%> (-1.71%)
44.00% <0.00%> (-1.00%)

...tadog/android/log/internal/domain/LogSerializer.kt
90.00% <0.00%> (-1.43%)
28.00% <0.00%> (ø%)

...g/android/rum/internal/domain/scope/RumEventExt.kt
87.64% <0.00%> (-1.12%)
0.00% <0.00%> (ø%)

...l/net/info/BroadcastReceiverNetworkInfoProvider.kt
81.05% <0.00%> (ø)
28.00% <0.00%> (ø%)

...android/rum/internal/ndk/DatadogNdkCrashHandler.kt
82.95% <0.00%> (+0.78%)
25.00% <0.00%> (+1.00%)

...c/main/kotlin/com/datadog/android/DatadogConfig.kt
94.90% <0.00%> (+1.02%)
12.00% <0.00%> (+2.00%)

...oid/src/main/kotlin/com/datadog/android/Datadog.kt
89.41% <0.00%> (+1.18%)
29.00% <0.00%> (+1.00%)


ssh-demo: Update runtimeClassName

Update runtimeClassName from kata-cc to kata to reflect the operator changes
Signed-off-by: stevenhorsman<EMAIL_ADDRESS>

@bpradipt - I think that we moved away from kata-cc to kata for the default runtime class name a while ago?

@stevenhorsman the ssh-demo uses the old operator code from ccv0-demo branch. The operator binary and the payload points to ccv0. Once the new ssh payload is there, we can update the instructions to use the latest operator code (deploy.yaml) and the payload.

Ah ok, maybe this wasn't a good change to make yet. I was hoping that we could re-use the kubernetes yaml file from this in the new documentation, but maybe we're not ready for that until the sample payload is available.

This seems good to me.
@bpradipt mentions something about the SSH demo using the old operator. I don't really remember how this worked. It seems like these instructions don't say anything about the operator. They just tell someone how to build a container image and deploy it. It seems like a good idea to use a runtime class that will work.

Jakob had built the initial ssh-demo payload using the first code drop for ccv0 branch. Also, the manifests used were specific to the old operator code (before v0.1.0). Now we have the latest ssh-demo payload built by Steve and it works with the latest operator code, so this change is good..


zend-db documentation addition

Provide a narrative description of what you are trying to accomplish:

[ ] Are you fixing a bug?

[ ] Detail how the bug is invoked currently.
[ ] Detail the original, incorrect behavior.
[ ] Detail the new, expected behavior.
[ ] Base your feature on the master branch, and submit against that branch.
[ ] Add a regression test that demonstrates the bug, and proves the fix.
[ ] Add a CHANGELOG.md entry for the fix.

[ ] Are you creating a new feature?

[ ] Why is the new feature needed? What purpose does it serve?
[ ] How will users use the new feature?
[ ] Base your feature on the develop branch, and submit against that branch.
[ ] Add only one feature per pull request; split multiple features over multiple pull requests
[ ] Add tests for the new feature.
[ ] Add documentation for the new feature.
[ ] Add a CHANGELOG.md entry for the new feature.

[ ] Is this related to quality assurance?

[x] Is this related to documentation?

This repository has been closed and moved to laminas/laminas-db; a new issue has been opened at https://github.com/laminas/laminas-db/issues/24.

This repository has been moved to laminas/laminas-db. If you feel that this patch is still relevant, please re-open against that repository, and reference this issue. To re-open, we suggest the following workflow:

Squash all commits in your branch (git rebase -i origin/{branch})
Make a note of all changed files (`git diff --name-only origin/{branch}...HEAD
Run the laminas/laminas-migration tool on the code.
Clone laminas/laminas-db to another directory.
Copy the files from the second bullet point to the clone of laminas/laminas-db.
In your clone of laminas/laminas-db, commit the files, push to your fork, and open the new PR.
We will be providing tooling via laminas/laminas-migration soon to help  automate the process.


reset active workspace before listing workspaces

Fixes

https://issues.redhat.com/browse/RHTAPBUGS-568
Description

Fetching workspaces returns empty array due to the invalid url.
RCA:
UI makes calls to list the workspaces and sets the active namespace and workspace, but for this call to work properly we should not set the workspace context prior to this call or else this will result in empty array and impact all other subsequent API calls that UI makes.
Solution:
Reset the active workspace before list workspaces
Type of change

[x] Bugfix

Screen shots / Gifs for design review

Before:

After:
https://github.com/openshift/hac-dev/assets/9964343/84dd0401-3d2e-4a5b-b1f5-3319af2ce784

How to test or reproduce?

Browser conformance:

[x] Chrome
[x] Firefox
[x] Safari
[ ] Edge

/retest

/retest

/retest

/retest

/lgtm

/retest


omnibus "too many open files" all-inclusive ticket

The following tickets are all related to "too many open files"
and can all be solved with smarter open fd management.
fd's can be leaked by open files or network sockets
https://github.com/NebulousLabs/Sia/issues/2202
https://github.com/NebulousLabs/Sia/issues/2236
https://github.com/NebulousLabs/Sia/issues/2243
https://github.com/NebulousLabs/Sia/issues/2272
https://github.com/NebulousLabs/Sia/issues/2288
https://github.com/NebulousLabs/Sia/issues/2307
https://github.com/NebulousLabs/Sia/issues/2326
https://github.com/NebulousLabs/Sia/issues/2363
@tbenz9 suggest the above linked open issues should be closed.

Excellent, thanks for making this.
Devs: Can we label this 'bug'

Still happening. Running Sia 1.3 on macOS Sierra (unable to get version information). When this error happens, I have to reboot my system to regain basic functionality. And each time this happens, I'm not running an excess number of applications. Typically my Ram and CPU usage, even with wallets running, is around 70% Ram and 50% CPU
Loading...
(0/6) Loading siad...
(1/6) Loading gateway...
(2/6) Loading consensus...
(3/6) Loading transaction pool...
(4/6) Loading wallet...
(5/6) Loading host...
(6/6) Loading renter...
Finished loading in 19.198039287 seconds
goroutine 1 [running]:
runtime/debug.Stack(0x1043947, 0x0, 0xc421eef5c0)
/usr/local/go/src/runtime/debug/stack.go:24 +0x79
runtime/debug.PrintStack()
/usr/local/go/src/runtime/debug/stack.go:16 +0x22
github.com/NebulousLabs/Sia/build.Critical(0xc421eef930, 0x1, 0x1)
/home/luke/go/src/github.com/NebulousLabs/Sia/build/critical.go:16 +0xa7
main.startDaemon(0x0, 0x0, 0x7fff5fbffca7, 0xe, 0x7fff5fbffcd3, 0x5, 0x7fff5fbffcc2, 0x5, 0x0, 0x7fff5fbffd2e, ...)
/home/luke/go/src/github.com/NebulousLabs/Sia/siad/daemon.go:311 +0xaa9
main.startDaemonCmd(0xc420093200, 0xc4200952c0, 0x0, 0x5)
/home/luke/go/src/github.com/NebulousLabs/Sia/siad/daemon.go:335 +0x162
github.com/spf13/cobra.(*Command).execute(0xc420093200, 0xc42006c070, 0x5, 0x5, 0xc420093200, 0xc42006c070)
/home/luke/go/src/github.com/spf13/cobra/command.go:653 +0x299
github.com/spf13/cobra.(*Command).ExecuteC(0xc420093200, 0x16fce88, 0x3b, 0xc420166aa0)
/home/luke/go/src/github.com/spf13/cobra/command.go:728 +0x339
github.com/spf13/cobra.(*Command).Execute(0xc420093200, 0x1aefba0, 0x16e58f7)
/home/luke/go/src/github.com/spf13/cobra/command.go:687 +0x2b
main.main()
/home/luke/go/src/github.com/NebulousLabs/Sia/siad/main.go:174 +0x7aa
Critical error: accept tcp <IP_ADDRESS>:9980: accept: too many open files in system
Please submit a bug report here: https://github.com/NebulousLabs/Sia/issues
Closing renter...

I have the same issue as @Wlinc
Same os, sia version, & error

@Wlinc @edendekker Would you please confirm if you're running Sia-UI when the issue occurs?

Indeed. Biggest question for us is whether the culprit is Sia-UI or siad. Historically, the problems have been Sia-UI not properly managing file descriptors as it opens connections to the API. But it's possible that siad is causing problems as well.

Hey @DavidVorick do you have any idea on https://github.com/NebulousLabs/Sia-UI/issues/752 , which is a place where fd leak happens for sure.

If that's the question it seems setting a lower limit than the system hard limit using ulimit -n may solve the problem.  You can for example limit each of siad and Sia-UI to half the file descriptors each and then you will (more likely) know which one is broken because it will not (immediately due to fd exhaustion) crash the other. @DavidVorick  @mckelvin

@mckelvin I am running Sia UI.

I haven't seen this pop up in nearly 3 months.  I'm going to close it due to inactivity.  We can re-open it if the error comes back.

This Problem KEEPS occuring.  You are closing a subject without resolving the probelm.
My Sia closes down once or twice a DAY ..>EVERYDAY!!!
This is wrecking my Uptime and my rating....Im losing Contracts.
If no Developer can resolve this problem, then Sia is DONE.

@taiwil I'm sorry you're having issues.  Can you provide more information about Sia is doing that is causing it to crash?  We believe this is an issue with the User Interface (UI), have you tried running Sia without the UI?
Lastly, the reason this issue is closed is that it's being tracked in https://github.com/NebulousLabs/Sia-UI/issues/752

@tbenz9
I am running IOS 10.11.6
I had followed created a file via the instructions here(which i think you may have linked me):
https://superuser.com/questions/433746/is-there-a-fix-for-the-too-many-open-files-in-system-error-on-os-x-10-7-1
I don't know if that file is effected when I repair my Disk permisssions.
In the terminal, I am increasing the Max number of open files.
I am still getting THIS Error:
Sia Daemon v1.3.2
Git Revision 72938f5
Loading...
(0/6) Loading siad...
(1/6) Loading gateway...
(2/6) Loading consensus...
(3/6) Loading transaction pool...
(4/6) Loading wallet...
(5/6) Loading host...
(6/6) Loading renter...
Finished loading in 20.749124299 seconds
goroutine 1 [running]:
runtime/debug.Stack(0x1045ec7, 0x0, 0xc420041948)
/usr/local/go/src/runtime/debug/stack.go:24 +0xa7
runtime/debug.PrintStack()
/usr/local/go/src/runtime/debug/stack.go:16 +0x22
github.com/NebulousLabs/Sia/build.Critical(0xc420041ac8, 0x1, 0x1)
/home/luke/go/src/github.com/NebulousLabs/Sia/build/critical.go:16 +0x93
main.startDaemon(0x0, 0x0, 0x7fff5fbffd53, 0xe, 0x7fff5fbffd7f, 0x5, 0x7fff5fbffd6e, 0x5, 0x0, 0x7fff5fbffdd3, ...)
/home/luke/go/src/github.com/NebulousLabs/Sia/cmd/siad/daemon.go:212 +0x588
main.startDaemonCmd(0xc4200acfc0, 0xc42007e550, 0x0, 0x5)
/home/luke/go/src/github.com/NebulousLabs/Sia/cmd/siad/daemon.go:236 +0x162
github.com/spf13/cobra.(*Command).execute(0xc4200acfc0, 0xc42000e0d0, 0x5, 0x5, 0xc4200acfc0, 0xc42000e0d0)
/home/luke/go/src/github.com/spf13/cobra/command.go:750 +0x2c1
github.com/spf13/cobra.(*Command).ExecuteC(0xc4200acfc0, 0x16e1b17, 0x3b, 0xc420099a40)
/home/luke/go/src/github.com/spf13/cobra/command.go:831 +0x30e
github.com/spf13/cobra.(*Command).Execute(0xc4200acfc0, 0x1b0a200, 0x16ca346)
/home/luke/go/src/github.com/spf13/cobra/command.go:784 +0x2b
main.main()
/home/luke/go/src/github.com/NebulousLabs/Sia/cmd/siad/main.go:174 +0x790
Critical error: accept tcp <IP_ADDRESS>:9980: accept: too many open files in system
Please submit a bug report here: https://github.com/NebulousLabs/Sia/issues
Shutdown complete.
I hope you know a solution.
Thank you for your reply,
Wil

@tbenz9
I don't know how to run Sia Without IU.   I host many files and contracts.

This issue is closed but I just got this error on Sia v1.3.3 on Linux. I think I found the culprit. I ran ulimit and got back unlimited. If someone is having this trouble then try setting ulimit to some reasonable value.


Zoom feature?

Is there any zoom feature (console or GUI button) for 2D /3D plots inside the matplotlib?

Yes, there should be a button with a magnifying glass symbol below the plot. (at least that's how it looks on my machine)


New version: Mega.MEGASync version <IP_ADDRESS>

Pull request has been created with Komac v1.11.0 :rocket:
Microsoft Reviewers: Open in CodeFlow

Automatic Validation ended with:

2024-02-07T04:28:01.3608103Z ##[error] Manifest Error: Duplicate installer entry found.

(Automated response - build 695.)

/AzurePipelines run

Automatic Validation ended with:

Installation failed with exit code -1978335226 2024-02-07 10:25:21.175 [CLI ] ShellExecute installer failed: 2

(Automated response - build 708.)

@wingetbot waivers Add Validation-Unattended-Failed

Hi @vvirtues,

This PR's version number 4.12.2 has 3 parameters (sets of numbers between dots - major, minor, etc), which is less than the current manifest's version <IP_ADDRESS>, which has 4 parameters.

Does this PR's version number exactly match the version reported in the Apps & features Settings page? (Feel free to attach a screenshot.)
(Automated response - build 720)

Hi @vvirtues,

This PR's version number 4.12.2 has 3 parameters (sets of numbers between dots - major, minor, etc), which is less than the current manifest's version <IP_ADDRESS>, which has 4 parameters.

Does this PR's version number exactly match the version reported in the Apps & features Settings page? (Feel free to attach a screenshot.)
(Automated response - build 720)

uh

This application doesn't write version data to the expected locations:

The EXE properties show a matching version:

But this has an extra .0, which has as many parameters as the previous version.

Do I move the entire version to a new folder?

or do I remove the .0 from certain manifest files?

Automatic Validation ended with:

2024-02-09T21:48:25.9583906Z ##[error] One or more errors occurred. (Manifest: The manifest file path must match PackageIdentifier and PackageVersion properties from manifest. actual 'manifests\m\Mega\MEGASync\4.12.2', expected 'manifests\m\Mega\MEGASync\<IP_ADDRESS>') Manifest: The manifest file path must match PackageIdentifier and PackageVersion properties from manifest. actual 'manifests\m\Mega\MEGASync\4.12.2', expected 'manifests\m\Mega\MEGASync\<IP_ADDRESS>'

(Automated response - build 726.)

Do I move the entire version to a new folder?

Or rename the folder.

haven't had time to look at it, will review over the weekend

Automatic Validation ended with:

2024-02-26 14:59:32.774 [CLI ] Installer [X64,nullsoft,User,en-US] not applicable: Machine is not compatible with X64

(Automated response - build 828.)

mightve updated

@wingetbot run

Automatic Validation ended with:

Installer Verification Analysis Context Information:
Specified hash doesn't match.
Actual hash:   CCE0585C865A68C4C124FA450A82831F163C7FDC3569AFD966F1CB30F87A963D
Expected hash: 73bcee878da54c3b37c2ee00f4b3e3070136ea1d2e3438a7299537fd5875af61
Specified hash doesn't match.
Actual hash:   73BCEE878DA54C3B37C2EE00F4B3E3070136EA1D2E3438A7299537FD5875AF61
Expected hash: cce0585c865a68c4c124fa450a82831f163c7fdc3569afd966f1cb30f87a963d

2/27/2024 5:06:52 PM

(Automated response - build 835.)


[Issue] - Health Checker - No KB or update detection

Version <IP_ADDRESS>3
The script works fine, at the first time.
I identified the vulnerabilities and installed the KB updates (OS and Exchnage).
Once the system rebooted, I ran the script again and the output tells me that the system is still vulnerable.
I've the same output, with Windows Server 2012 R2 with Exchange Server 2013, and Windows Server 2019 and Exchnage Server 2019.

What is the vulnerability that it is reporting? Some vulnerabilities, you need to actually take action on besides just installing a patch.

CVE-2022-24516, CVE-2022-21979, CVE-2022-21980, CVE-2022-24477, CVE-2022-30134
fixed by: KB5019076 (Exchange 2013 fix, installed)
CVE-2020-1147: fixed by: KB5021294 (OS rollup .net , installed)

@land0sysadm  these CVEs require Extended Protection to also be configured, not just the SU installed.
CVE-2022-24516, CVE-2022-21979, CVE-2022-21980, CVE-2022-24477, CVE-2022-30134
The CVE-2020-1147, is just a .NET security update. I would like to see the xml output from the Health Checker to see if anything is wrong with this check, as looking at the code there could be some flaws. Please send the data to<EMAIL_ADDRESS>

@land0sysadm Can you please provide the output? I have not seen them yet and I would like to make sure that we address this issue. Otherwise, I will have to close out the issue.

@land0sysadm Can you please provide the output? Otherwise, I will close out the issue.

No output provided. Will close this issue for now.


The icons in the footer are not aligned in the center of page

Describe the bug
The icons in the footer are not aligned in the center
Expected behavior
The icons would be in the center of the page
Screenshots

Desktop (please complete the following information):

OS: Windows
Browser Edge

I found out the problem can I work on this issue?

sure @Shalini469717 you can make a pull request of this issue

yes!

#4

#5

completed by @Shalini469717


removing 1 SU note

#sign-off

@JSeb225 : Thanks for your contribution to the Azure documentation! The author, @JSeb225, has been notified to review your proposed change.

Microsoft authors of articles need to make updates through the private repository so the updates can be staged and validated by the current validation rules. After you review staging and fix all validation issues, you can add the sign-off comment to let the PR reviewers know the updates are ready to be merged.
It's easy to access the private repo version of your articles in GitHub. Just add -pr to the URL, as follows:
Public repo URL: https://github.com/Microsoft/azure-docs/blob/master/articles/azure-portal-overview.md
Private repo URL: https://github.com/Microsoft/azure-docs-pr/blob/master/articles/azure-portal-overview.md
We've closed this pull request. Please resubmit the updates through the private repo. If you need perms to the private repo, see the contributor guide instructions


Multi GPU train of flux Have Some Bugs

Multi GPU train flux, Is the script of flux train not supported now?

It supports on sd-3 branch. If you have trouble logs, please attach them.

Now this exception occurs during accelerator prepare, but when I modify it to be the same as sdxl, there are other problems at the dataloader.

It supports on sd-3 branch. If you have trouble logs, please attach them.

Now this exception occurs during prepare, but when I modify it to be the same as sdxl, there are other problems at the dataloader.

Unfortunately multi GPU training of FLUX has not been tested yet. --split_mode doesn't seem to work with multi GPU training.

The current single-card training is indeed too slow for flux, especially for fine-tuning at the level of pony or animation. It is beneficial to the open source community to prioritize fixing multi-GPU problems.

Unfortunately multi GPU training of FLUX has not been tested yet. --split_mode doesn't seem to work with multi GPU training.

The current single-card training is indeed too slow for flux, especially for fine-tuning at the level of pony or animation. It is beneficial to the open source community to prioritize fixing multi-GPU problems.

In my toy scripts, training Flux with FSDP in accelerator works.
used accelerate configuration and some code fixing
fsdp_config.yaml
compute_environment: LOCAL_MACHINE
distributed_type: FSDP
downcast_bf16: 'no'
fsdp_config:
  fsdp_auto_wrap_policy: SIZE_BASED_WRAP
  fsdp_backward_prefetch_policy: BACKWARD_PRE
  fsdp_forward_prefetch: true
  fsdp_offload_params: false
  fsdp_sharding_strategy: 1
  fsdp_state_dict_type: FULL_STATE_DICT
  fsdp_sync_module_states: true
  fsdp_use_orig_params: true
  fsdp_min_num_params: 100000000
machine_rank: 0
main_training_function: main
mixed_precision: bf16
num_machines: 1
num_processes: 4
rdzv_backend: static
same_network: true
tpu_env: []
tpu_use_cluster: false
tpu_use_sudo: false
use_cpu: false

in flux_train.py Line 182
from
train_dataset_group.new_cache_latents(ae, accelerator.is_main_process)
to
train_dataset_group.new_cache_latents(ae, True)
If you use caching latents or cached output of text encoder.
And sample scripts here,
accelerate launch --config_file=$ACCELERATE_CONFIG \
    --num_processes=$DEVICE_COUNT --num_machines=$NUM_MACHINES --mixed_precision=$MIXED_PRECISION \
    --main_process_ip=$MAIN_PROCESS_IP --main_process_port=$MAIN_PROCESS_PORT \
    --num_cpu_threads_per_process=2 \
    flux_train.py --config_file=$CONFIG_FILE

Updated the sd3 branch. Multi-GPU training should now work.
Kohya you are a God, thank you for the multi-GPU update!!

Updated the sd3 branch. Multi-GPU training should now work.

accelerate launch --mixed_precision bf16 --num_cpu_threads_per_process 1 flux_train.py  --pretrained_model_name_or_path /workspace/pretrain_model/flux1-dev.safetensors  --clip_l /workspace/pretrain_model/clip_l.safetensors --t5xxl /workspace/pretrain_model/t5xxl_fp16.safetensors --ae /workspace/pretrain_model/ae_dev.safetensors --save_model_as safetensors --sdpa --persistent_data_loader_workers --max_data_loader_n_workers 2 --seed 42 --gradient_checkpointing --mixed_precision bf16 --save_precision bf16   --output_dir /workspace/sa_poc --output_name sa --learning_rate 7.5e-6 --max_train_epochs 10  --sdpa --highvram --cache_text_encoder_outputs_to_disk --cache_latents_to_disk --save_every_n_epochs 1 --optimizer_type adafactor --optimizer_args "relative_step=False" "scale_parameter=False" "warmup_init=False" --timestep_sampling sigmoid --model_prediction_type raw --guidance_scale 1.0 --blockwise_fused_optimizers  --double_blocks_to_swap 6 --cpu_offload_checkpointing  --train_data_dir /workspace/sa_poc/sa --in_json /workspace/sa_poc/lat_small.json --resolution "1024,1024" --train_batch_size 42
I run this command but got an error

Updated the sd3 branch. Multi-GPU training should now work.更新了 sd3 分支。多 GPU 训练现在应该可以工作了。

When I use instructions related to double_blocks, an error will be reported. After canceling, it can run, but the GPU utilization is unstable, and when the batch_size is 4, the memory usage is already very large.

my command
accelerate launch --mixed_precision bf16 --num_cpu_threads_per_process 1 flux_train.py --pretrained_model_name_or_path /workspace/pretrain_model/flux1-dev.safetensors --clip_l /workspace/pretrain_model/clip_l.safetensors --t5xxl /workspace/pretrain_model/t5xxl_fp16.safetensors --ae /workspace/pretrain_model/ae_dev.safetensors --save_model_as safetensors --sdpa --persistent_data_loader_workers --max_data_loader_n_workers 2 --seed 42 --gradient_checkpointing --mixed_precision bf16 --save_precision bf16 --output_dir /workspace/sa_poc --output_name sa --learning_rate 7.5e-6 --max_train_epochs 10 --sdpa --highvram --cache_text_encoder_outputs_to_disk --cache_latents_to_disk --save_every_n_epochs 1 --optimizer_type adafactor --optimizer_args "relative_step=False" "scale_parameter=False" "warmup_init=False" --timestep_sampling sigmoid --model_prediction_type raw --guidance_scale 1.0 --cpu_offload_checkpointing --train_data_dir /workspace/sa_poc/sa --in_json /workspace/sa_poc/lat_small.json --resolution "1024,1024" --train_batch_size 4
or can you give me a command for test

--double_blocks_to_swap and --single_blocks_to_swap cannot be used with multi-GPU training.
--cpu_offload_checkpointing reduces the memory usage, but the GPU utilization seems to be reduced. You may want to remove that option (and decrease the batch size if needed).

thank you for your answer
fp16 can run, but loss is nan, accelerate also set up fp16

hi @kohya-ss , can I use this feature on both Windows and Linux now? Have you finished developing it?

hi @kohya-ss
I encountered this error:
train_network.py", line 1088, in train     text_encoder_conds is None UnboundLocalError: local variable 'text_encoder_conds' referenced before assignment

Has anyone managed to get it working properly? I have 4 GPUs, but it still only runs on the first GPU.

Has anyone managed to get it working properly? I have 4 GPUs, but it still only runs on the first GPU.

Is trainer in caching process?

Has anyone managed to get it working properly? I have 4 GPUs, but it still only runs on the first GPU.

Is trainer in caching process?

I don't think so, but is there a way to check this? @BootsofLagrangian

See terminal you run script, you can find trainer in caching processing or not.
And check your accelerate configuration contains distributed mode is MULTI_GPU and num_processes is number of gpu, in ~/.cache/huggingface/accelerate/default_config.yaml

hi @BootsofLagrangian
Could you please share your accelerate config?
And an example of the run script for training? Thank you so much!

hi @BootsofLagrangian I am using Windows. Could you please share your accelerate config? And an example of the run script for training? Thank you so much!

Here are a configuration of accelerate for 4-GPUs with DDP on bf16 mixed precision.
compute_environment: LOCAL_MACHINE
debug: true
distributed_type: MULTI_GPU
downcast_bf16: 'no'
enable_cpu_affinity: false
gpu_ids: all
machine_rank: 0
main_training_function: main
mixed_precision: bf16
num_machines: 1
num_processes: 4
rdzv_backend: static
same_network: true
tpu_env: []
tpu_use_cluster: false
tpu_use_sudo: false

If you want to use it more flexibly, save the above content as ddp.yaml, and then pass it as an argument to accelerate launch like so: accelerate launch --config_file='ddp.yaml' flux_train_network.py --config_file=SCRIPT_FOR_CONFIG.yaml.
And an example config and script for training flux with lora(if you want full finetuning Flux use flux_train.py instead of flux_train_network.py)
pretrained_model_name_or_path = "/path/to/models/sd-3/Flux.1-dev/flux1-dev.safetensors"
clip_l = "/path/to/models/sd-3/Flux.1-dev/clip_l.safetensors"
t5xxl = "/path/to/models/sd-3/Flux.1-dev/t5xxl_fp16.safetensors"
ae = "/path/to/models/sd-3/Flux.1-dev/ae.safetensors"

timestep_sampling = "sigmoid"
model_prediction_type = "raw"
guidance_scale = 1.0
loss_type = "l2"

network_module = "networks.lora_flux"
network_dim = 16
network_alpha = 16

save_state = true
cache_info = true
sdpa = true
fp8_base = true
highvram = true
seed = 42

mixed_precision = "bf16"
save_precision = "bf16"
t5xxl_dtype = "fp16"

output_name = "YOUR_OUTPUT_NAME"
output_dir = "/path/to/output"
logging_dir = "/path/to/logging"
train_data_dir = "/path/to/dataset"
save_every_n_epochs = 1
save_every_n_steps = 100

cache_latents = true
cache_latents_to_disk = true
cache_text_encoder_outputs = true
cache_text_encoder_outputs_to_disk = true

caption_extension = ".txt"
caption_separator = ". "

vae_batch_size = 4

resolution = "1024,1024"
enable_bucket = true
bucket_no_upscale = true

train_batch_size = 4
gradient_accumulation_steps = 1
max_train_epochs = 10
max_data_loader_n_workers = 1
persistent_data_loader_workers = true
gradient_checkpointing = true

learning_rate = 3e-4
network_train_unet_only = true

max_grad_norm = 1.0
optimizer_type = "adamw8bit"
save_model_as = "safetensors"
optimizer_args = [ "weight_decay=1e-1", "betas=(0.95, 0.98)"]
lr_scheduler = "cosine"
lr_warmup_steps = 50

This is an example.yaml file. With this example file run in virtual envrionment, accelerate launch --config_file=ddp.yaml flux_train_network.py --config_file=example.yaml
You should change script configuration for your own path and dataset.

i will try
Thank you so much! @BootsofLagrangian

@BootsofLagrangian You mention swapping flux_train.py for flux_train_network.py to do multi-GPU full finetune.
With your config, I can run flux_train_network.py with no problems, but flux_train.py throws an out-of-memory error. Watching nvidia-smi, I can see that there is no VRAM usage on the second GPU before this happens, it doesn't seem to be spreading shards across the two GPUs and instead trying to load everything into the first one.
I have tried this with and without the code change to flux_train.py that you suggested in an earlier comment.
I have two RTX 4090s, and I'm not sure if this is enough to finetune flux without quantization or block swapping, but it seems like it's not working as intended if it's throwing OOM errors without using the second GPU at all.
Were you able to get flux_train.py to work with the config you shared?

--config_file=example.yam

I used this configuration and parameters, but still got an error. This has been bothering me for a long time. How can I solve it
Traceback (most recent call last):
File "e:\pinokio\bin\miniconda\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "e:\pinokio\bin\miniconda\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "E:\pinokio\api\fluxgym.git\env\Scripts\accelerate.exe_main.py", line 7, in
File "e:\pinokio\api\fluxgym.git\env\lib\site-packages\accelerate\commands\accelerate_cli.py", line 48, in main
args.func(args)
File "e:\pinokio\api\fluxgym.git\env\lib\site-packages\accelerate\commands\launch.py", line 1159, in launch_command
multi_gpu_launcher(args)
File "e:\pinokio\api\fluxgym.git\env\lib\site-packages\accelerate\commands\launch.py", line 793, in multi_gpu_launcher
distrib_run.run(args)
File "e:\pinokio\api\fluxgym.git\env\lib\site-packages\torch\distributed\run.py", line 910, in run
elastic_launch(
File "e:\pinokio\api\fluxgym.git\env\lib\site-packages\torch\distributed\launcher\api.py", line 138, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "e:\pinokio\api\fluxgym.git\env\lib\site-packages\torch\distributed\launcher\api.py", line 260, in launch_agent
result = agent.run()
File "e:\pinokio\api\fluxgym.git\env\lib\site-packages\torch\distributed\elastic\metrics\api.py", line 137, in wrapper
result = f(*args, **kwargs)
File "e:\pinokio\api\fluxgym.git\env\lib\site-packages\torch\distributed\elastic\agent\server\api.py", line 696, in run
result = self._invoke_run(role)
File "e:\pinokio\api\fluxgym.git\env\lib\site-packages\torch\distributed\elastic\agent\server\api.py", line 849, in _invoke_run
self._initialize_workers(self._worker_group)
File "e:\pinokio\api\fluxgym.git\env\lib\site-packages\torch\distributed\elastic\metrics\api.py", line 137, in wrapper
result = f(*args, **kwargs)
File "e:\pinokio\api\fluxgym.git\env\lib\site-packages\torch\distributed\elastic\agent\server\api.py", line 668, in _initialize_workers
self._rendezvous(worker_group)
File "e:\pinokio\api\fluxgym.git\env\lib\site-packages\torch\distributed\elastic\metrics\api.py", line 137, in wrapper
result = f(*args, **kwargs)
File "e:\pinokio\api\fluxgym.git\env\lib\site-packages\torch\distributed\elastic\agent\server\api.py", line 500, in _rendezvous
rdzv_info = spec.rdzv_handler.next_rendezvous()
File "e:\pinokio\api\fluxgym.git\env\lib\site-packages\torch\distributed\elastic\rendezvous\static_tcp_rendezvous.py", line 67, in next_rendezvous
self._store = TCPStore(  # type: ignore[call-arg]
RuntimeError: use_libuv was requested but PyTorch was build without libuv support

accelerate launch --config_file=ddp.yaml flux_train_network.py --config_file=example.yaml

you are my hero, bro.
this script can running on multi-gpu service, about 18G memory


Add lua support

Add lua.scm from nvim-treesitter
Finish lua support
Revert "Add lua.scm from nvim-treesitter" for some reason it's causing the program to panic

I'll take a look on why adding the highlights is causing the program to panic

@Wilfred Are you going to squash merge this? or should I squash the last few commits?

@JafarAbdi please squash the last few commits, and then I will merge this :)

Thanks :)


Add Highlight Support in PPL and Wildcard in SQL and PPL

Description
Support the highlight function in PPL. Add support for wildcard in SQL and PPL.
Issues Resolved
Issue: github:636
Check List

[x] New functionality includes testing.

[x] All tests pass, including unit test, integration test and doctest

[x] New functionality has been documented.

[x] New functionality has javadoc added
[ ] New functionality has user manual doc added

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@forestmvey I'd like to have a second look at Object.toString usages.
I understand that in context it provides required data, but semantically toString is intended for something else. This is what the docs have to say about it.

Can you please be more specific and give an example of what you would like revised.

Can you please be more specific and give an example of what you would like revised.
This is the part I was thinking of.

Also, based on this comment, it'd be better if HighlightOperator moves to :opensearch module.
That should also allow us to use OpenSearch's Regex class and make pattern matching consistent.


[FEATURE] async support

Is your feature request related to a problem? Please describe.
dag execute async functions
Describe the solution you'd like
could decorate async function
the decorator support passing in pool executor in case for running sync

Can you please describe the purpose of doing this ?
The goal of tawazi is to paralellize the execution of sync code. Are you forced to use async code ?

we use fastapi async mode, each node in the execution graph just a http request to other service. using async will be of great help.

this feature is not in the roadmap sadly.
But if you transform each node to sync, tawazi wil run each node in a separate thread. This would be helpful because your http requests won't block the Python interpreter. This should achieve the results you want.
However I still don't understand why you want to use tawazi in your case... if your code is already completely async, you don't need tawazi... However, if some parts of your code are async and others are sync tawazi is useful for you.

we are working on async support here: https://github.com/mindee/tawazi/pull/217
It doesn't support async methods yet but transforms sync functions / ExecNodes into awaitables.
Can this be useful to your use case ?
Currently we don't have the incentive to do async support. It could be implemented with ease now though

done in tawazi 0.4


Different zoom levels / fontsizes per editor group

I often like to have files that I only need to look at in a separate (tiny) editor group (see picture, groups on the right), but this means that I can only see a very small portion of the code.
It would be useful to set a smaller font size for these editor groups, and still have the normal font size in the main editor group.

Related on Stack Overflow: How can I zoom one single editor group independently from others in VS Code?


Remove NavPi store from Merchants

[ ] Description is satisfied.
[ ] Changes display correctly on desktop / mobile.
[ ] Translations have been added or amended as necessary.

Bounty Low shared by issues #218 #217 #216 #215 #152

Claimed.


responsive mode not availabe

mode as shown in tutorial is not there

Right click on the ruler 📏  corner and you will see a context menu. Choose 'show responsive control' to bring up the responsive mode.

Good morning
I have done it but tool bars in graphic mode are note responding for eg
1 missing div tool
2 quick responsive layout tool from code side bar as seen in ur tutorial
video all functions are missing
I don't know why?
Please help me out
Regards
Areeb
On May 14, 2016 10:06 PM, "Swagatam Mitra"<EMAIL_ADDRESS>wrote:

Right click on the ruler 📏 corner and you will see a context menu.
Choose 'show responsive control' to bring up the responsive mode.
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/swmitra/html-designer/issues/7#issuecomment-219229913

Did you ever get an answer to this?


Support staking for Creditcoin

Describe the task
Support staking for Creditcoin: https://staking.creditcoin.org/#/overview

Version test: https://file.cdmteck.com/s/4oimk9zDJ8R5Aai/download?path=%2F&files=./upgrade-ui-pr-1884-build-1.1.10-2023-09-08T10-16-13UTC.zip
Validation passed for cases:

Show staking item
Network and validator information
Stake more/ unstake (Approve step with QR-signer account)


Fix some issues with supplier barcode plugins

I went through a bunch of parts with the new barcode scanning functionality again and noticed some bugs / possible improvements:

the most obvious one is in get_supplier_parts(...) in mixins.py (accidentally doing a fresh query there instead of reusing the existing one)
the digikey plugin should only handle barcodes which contain a SKU (the issue here is that digikey/mouser barcodes look identical, so for mouser barcodes, the digikey plugin tries to parse the barcode and calls get_supplier_parts(...), makes a bunch of queries and fails, then the mouser one gets called and does the same queries again)
if any plugin finds more than one valid SupplierPart via get_supplier_parts I return an error response, which leads to that error being directly returned to the end user. i think the better behavior would be to let the other plugins run first and only return the error if all of them fail (don't return a result) aswell. alternatively we could just get rid of said error response and return None instead
unused imports for logging in all supplier plugins (forgot to remove these apparently)

(This should be solved before a potential 0.13.0 release imo).

@30350n thanks for pushing these. I was playing around with barcodes yesterday and noticed some issues. I started a PR here - https://github.com/inventree/InvenTree/pull/5914 - but I think that I will close that one and try a different approach.
I noted that some digikey barcodes use 30P as the header for SKU. Have you seen this?

I noted that some digikey barcodes use 30P as the header for SKU. Have you seen this?

Don't think so, can you share one of those barcodes?

I started a PR here - https://github.com/inventree/InvenTree/pull/5914 - but I think that I will close that one and try a different approach.

Interesting, that seems like a larger refactor. Honestly I'm not super sure about the code structure we ended up with ... The main idea with having a plugin per supplier was to create a starting point for further supplier specific integrations, like search/part importing (i.e. directly integrating stuff like my part importing CLI into InvenTree.
Not sure how the roadmap looks like for that though exactly. If that's not going to be a thing (in core), then it might be better to refactor everything into one single big plugin again, which would be able to detect all the different supplier barcode formats and handle them accordingly.

@30350n thanks for the updates. I'll ping you when I make a new PR


Update fs2-core to 1.0.0-RC2

Updates co.fs2:fs2-core from 1.0.0-RC1 to 1.0.0-RC2.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention @scala-steward in the comments below.
Have a nice day!

Codecov Report

Merging #93 into master will increase coverage by 0.86%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master      #93      +/-   ##
==========================================
+ Coverage   96.52%   97.39%   +0.86%
==========================================
  Files          12       12
  Lines         115      115
==========================================
+ Hits          111      112       +1
+ Misses          4        3       -1

Impacted Files
Coverage Δ

...main/scala/com/github/gvolpe/fs2rabbit/model.scala
100% <0%> (+3.57%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c40c387...44e7217. Read the comment docs.


Bosch EasyControl Adapter

I'm new to this, and I cannot get on the domoticz forum because a cookie policy I need to accept that isn't visible to me. But there is an adapter for older thermostats that has EMS on the thermostat side and connects to the older heating system. (I assume the protocol of interest on that side is called iRT)
This makes my Buderus logamax U122 compatible with the Bosch EasyControl CT200, which according to the documentation, is using EMS.
So that might extent the usability of this project to more heating systems.

That adapter is mainly used to allow the Easy thermostat control older boilers. Although I have not tested it, I don't think it will convert (a lot of) messages from the boiler to the bus itself.
Perhaps you can try to see which datagrams are exactly transmitted on the bus. If there is a message type 0x18 UBAMonitorSlow observed it would make it compatible with the main sketch on my EMS Github page.


CLN: Datetimelike._can_hold_na

[x] tests added / passed
[x] passes git diff upstream/master | flake8 --diff
[x] no whatsnew, as it is internal fix

datetimelike Index._can_hold_na is all False even though these can store NaT.
idx = pd.DatetimeIndex(['2011-01-01', 'NaT'])
idx.hasnans
# True

idx._can_hold_na
# False

idx._nan_idxs
# array([], dtype=int64)

Because of test_base.py condition below, datetime-like Indexes had skipped some tests until now.

https://github.com/pydata/pandas/blob/master/pandas/tests/test_base.py#L219

Current coverage is 85.28% (diff: 100%)

Merging #13983 into master will decrease coverage by <.01%

@@             master     #13983   diff @@
==========================================
  Files           139        139
  Lines         50206      50212     +6
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
+ Hits          42818      42823     +5
- Misses         7388       7389     +1
  Partials          0          0

Powered by Codecov. Last update 29d9e24...8fdc0d1

Updated not to rely on #13979. _get_unique_index can be removed after #13984.

@jreback @jorisvandenbossche can u check? This is needed to make some Index dtype related fixes.

This is looking good to me. @jreback OK?

needs #13979 first

@sinhrks updated to not rely on that one

@jreback Now this PR doesn't rely on #13979.

@sinhrks This needs a rebase now I merged the other

lgtm. ping on green.

fixed lint error, and now green.

Thanks!


Implement Search 2

Implement second randomly assigned search method

[ ] Implement the method in UsersDatabase.
[ ] Write unit tests.


Undefined links should stop project from creation

In docker-compose, if an undefined link is provided to service.
It is not created. Similar behaviour should be for libcompose.
Signed-off-by: Kunal Kushwaha<EMAIL_ADDRESS>

Fix for https://github.com/docker/libcompose/issues/288

Seems good to me, @joshwget WDYT ?

Yeah, LGTM too

LGTM it is :angel:


[Event Change Request] - SalesLine.Table.al - OnCheckWarehouseOnAfterSetLocation2

We like to request the following event change:
Object: SalesLine.Table.al
ToDo: add Rec to parameter
Where is the event placed:
local procedure CheckWarehouse()
var
    ...
begin
    ...
    GetLocation("Location Code");
    if "Location Code" = '' then begin
        WhseSetup.Get();
        Location2."Require Shipment" := WhseSetup."Require Shipment";
        Location2."Require Pick" := WhseSetup."Require Pick";
        Location2."Require Receive" := WhseSetup."Require Receive";
        Location2."Require Put-away" := WhseSetup."Require Put-away";
    end else
        Location2 := Location;
    OnCheckWarehouseOnAfterSetLocation2(Rec, Location2);

    DialogText := Text035;
    ...
end;

Event Code:
[IntegrationEvent(false, false)]
local procedure OnCheckWarehouseOnAfterSetLocation2(var SalesLine: Record "Sales Line"; var Location2: Record Location)
begin
end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


fix metadata disable full scan

The changes in this PR address, but don't fix directly, this issues:

https://github.com/IBM/sarama/issues/2069
https://github.com/IBM/sarama/issues/2630

Also this will be of great help in Dapr: https://github.com/dapr/components-contrib/issues/3247
With this changes now it will be possible to set config.Metadata.Full = false but it would require the user to set a topic via
config.Metadata.InitialTopics = []string{"dummy-topic"} , with this the first request to fetch the metadata will either create that topic if it doesn't exist or it will just fetch its metadata

ping, is there any interest in this contribution? I'm happy to debate it further or change anything that is needed


Final Issue

For my final project with Hayden Chen we will make a ...  Virtual Pet Simulator
Use list comprehensions to manage lists of virtual pets based on attributes like species, age, or happiness level.
Process pet actions using conventional methods or forEach loops to simulate feeding, playing, and caring for virtual pets.
Implement sorting options for pets, such as sorting by species, age, or health status.
Include a search feature to allow users to find pets by name or species.
Use 2D iteration to display virtual pet habitats and interactions in a visually appealing manner.

I concur with this brilliant plan. I think it is most magnificent, truly a visionary mind.


Change  ActiveDirectory port to 636

Someone in my team write the following code:
let config = {
url: ldap://${ldapConnector.settings.address},
baseDN: this._renderDC(ldapConnector.settings.username),
username: ldapConnector.settings.username,
password: tools.decryptSymmetric(ldapConnector.settings.password),
port: ldapConnector.settings.port ,
tls: ldapConnector.settings.useSSL || false,
tlsOptions: {
rejectUnauthorized: true
}
};
let activeDirectory = new ActiveDirectory(config);
When ldap port is 389,  the authenticate function works fine,
but when the port is 636 or something else, it's seem like authenticate function try to pass throw port 389.
I understood  that activedirectory controller is not expect to port parameter,
I tried put in url: 'ldap://address:port", but authenticate function  throw the error:
Error: read ECONNRESET
at TCP.onStreamRead (internal/stream_base_commons.js:209:20)
So how can I change the port from default value?
Thanks,

when connect by 636 port (with ssl) change url to ->url: 'ldaps://address:port"  ldaps not ldap,
Now it's works!


Devicelab memory usage probes aren't working on iOS

broken probes

average_memory_usage
90th_percentile_memory_usage
99th_percentile_memory_usage

reproduction steps

Checkout flutter/flutter@0198afa1a4407e6b16ac0e57f6f23adc4e05b706
Modify dev/benchmarks/macrobenchmarks/ios/Runner.xcodeproj to show the graphics overlay (showGraphicsOverview)
Run the backdrop_filter_perf_ios__timeline_summary benchmark
Look at the overlay and notice that the memory usage is around 400MB
Wait for the results to be printed to the console, notice that it reports something around 200MB
Checkout flutter/flutter@def5a38e02fbec5a1737a83775ffefcc7f98eea9
Run the test again
Notice that the overlay now reports usage is around 200MB but the report printed to the console still lists the same result from above.

screenshots

they're not implemented for iOS, but we should!

they're not implemented for iOS, but we should!

The memory probes are implemented, they are the ones listed in the description, but they give bogus results.  I think you are talking about tracking gpu memory.


@types/react

Not able to install packages with non-standard names, example:
https://www.npmjs.com/package/@types/react
This is how your page looks like (it is broken!):
https://asset-packagist.org/package/detail?fullname=npm-asset/%40types/react

This npm feature is called scoped packages.
Please see this issue: https://github.com/hiqdev/asset-packagist/issues/54
I close this issue. If you still have questions please open new issue in asset-packagist package.


Update JDKs using which the project is built

Suggested commit message:
Update JDKs using which the project is built (#545)

- Perform a regular build using JDK 20 instead of JDK 19.
- Perform an experimental build using JDK 21-ea instead of JDK 20-ea.

The JDK 21-ea build fails due to an incompatibility with Error Prone 2.18.0. Requires a closer look.

When locally building against the latest revision of Error Prone, the JDK 21-ea errors seen here go away. But then the build fails due to an incompatibility in NullAway:
error-prone version: HEAD-SNAPSHOT
BugPattern: (see stack trace)
Stack Trace:
java.lang.NoClassDefFoundError: com/sun/tools/javac/code/TypeMetadata$Entry
    at com.uber.nullaway.NullAway.matchParameterizedType(NullAway.java:682)
    at com.google.errorprone.scanner.ErrorProneScanner.processMatchers(ErrorProneScanner.java:449)
    at com.google.errorprone.scanner.ErrorProneScanner.visitParameterizedType(ErrorProneScanner.java:783)
    at com.google.errorprone.scanner.ErrorProneScanner.visitParameterizedType(ErrorProneScanner.java:150)
    at jdk.compiler/com.sun.tools.javac.tree.JCTree$JCTypeApply.accept(JCTree.java:2825)
    at jdk.compiler/com.sun.source.util.TreePathScanner.scan(TreePathScanner.java:92)

I forgot that I had opened this PR; in the mean time I filed #604. Will close this one,


Nest foreign key relations

cc @awakia  @dtan4 @munisystem
WHY
https://geemus.gitbooks.io/http-api-design/content/en/responses/nest-foreign-key-relations.html

Summary Goes Here
Serialize foreign key references with a nested object, e.g.:
{
  "name": "service-production",
  "owner": {
    "id": "5d8201b0..."
  },
  // ...
}

Instead of e.g.:
{
  "name": "service-production",
  "owner_id": "5d8201b0...",
  // ...
}

This approach makes it possible to inline more information about the related resource without having to change the structure of the response or introduce more top-level response fields, e.g.:
{
  "name": "service-production",
  "owner": {
    "id": "5d8201b0...",
    "name": "Alice",
    "email"<EMAIL_ADDRESS>  },
  // ...
}

WHAT
Then now apig works this.
{
  "name": "service-production",
  "owner_id": "5d8201b0...",
  "owner": null
}

so I think apig should work in this way.
{
  "name": "service-production",
  "owner": {
    "id": "5d8201b0...",
  }
}

I want to hear everyone's opinion.
HOW

How to identify foreign key ?

gorm has gorm:"ForeignKey:Profile" tag.
But gorm works json:profile_id tag as gorm:"ForeignKey:Profile" without gorm:"ForeignKey:Profile".
ProfileID   uint      `json:"profile_id"`
// => ProfileID   uint      `json:"profile_id" gorm:"ForeignKey:Profile"`

So I don't know what is better that gorm:"ForeignKey:XXX" was mandatory.

gorm:"ForeignKey:XXX" was mandatory

No way.
Since gorm can understand that relation, we can understand it as well.

I want to hear everyone's opinion.

I like this idea, but it's not high priority.
This api is a kind of simple replacement of direct access of SQL database, so current response is not too bad.
If you can find simple and clear implementation to achieve this, I'm positive. Please keep in mind that you should not make the code too dirty to achieve this.


context.py:83: Split to ProductImagesContext and...

The puzzle<PHONE_NUMBER>2 from #550 has to be resolved:
https://github.com/fidals/refarm-site/blob/471fe2d1d8b6867da6567126cd1cc6f242764887/catalog/context.py#L83-L83
The puzzle was created by duker33 on 17-Sep-18.
Estimate: 30 minutes,
If you have any technical questions, don't ask me, submit new tickets instead. The task will be "done" when the problem is fixed and the text of the puzzle is removed from the source code. Here is more about PDD and about me.

The puzzle<PHONE_NUMBER>2 has disappeared from the source code, that's why I closed this issue.


Fix conversion test

A couple of things wrong here. You needed to include the install package
so that conversions that you wrote are loaded. You also can't encode
using the kapi scheme, because we use the configapi scheme.

awesome, thanks


RTMP/RTMPS streams play failure

could you please provide example url?

I am using my custom RTMP(S) URL and it is working for all the other players vlc,ffplay and mdk-sdk , i have tested thoroughly and ffmpeg is compiled with openssl , the problem is not my url but rather qtavplayer

test with nginx rtmp module

[h264 @ 0x7f1c3800aec0] co located POCs unavailable

rtmp and rtmps streams are now playing , really fast!, please close the issue.


Fixes #2110 Logging inconsistencies fixed in alerts.py

Removed the import of the logging lib and all the usages since they were not used properly and hid some errors. Changed the previous logger invocations to the proper logger elastalert_logger.

@Salaander
There is a description that seems to be applicable in the following file, but does it mean that the same correction is necessary?
elastalert/elastalert.py
elastalert/kibana_discover.py
elastalert/loaders.py
elastalert/opsgenie.py
elastalert/util.py

@Salaander
There is a description that seems to be applicable in the following file, but does it mean that the same correction is necessary?
elastalert/elastalert.py
elastalert/kibana_discover.py
elastalert/loaders.py
elastalert/opsgenie.py
elastalert/util.py


Storyblok plugins

Description
How can we extend Storyblok functionality by building plugins? How does one even build Storyblok plugins?
Acceptance criteria

[ ] What should we need a plugin for?
[ ] How do you build a plugin?

https://www.storyblok.com/docs/create-apps

https://www.storyblok.com/docs/plugins/plugin-storyblok-mixin#properties


Add auth reversal capability to Fusebox gateway

:shipit:

@andys I take back my previous comment - I see fusebox has both a "Void Sale" and an "Auth Reversal" transaction type.  I suppose both methods are necessary...


Build multi-arch images

Description
I've added make commands for building multi-arch docker image.
Removed artifacts/operator-<arch>.yaml because now you only need one deployment file for all archs.
I've update inlets image version and also bumped inlets-operator version so I guess multi-arch 0.2.7 image should be tagged before this gets merged? Or should I split this in two PRs?
How Has This Been Tested?

setup k3s cluster on arm64 server
deployed inlets-operator with docker imagematevzmihalic/inlets-operator:0.2.7
exposed deployment using DigitalOcean for the exit node

How are existing users impacted? What migration steps/scripts do we need?
There is a new unified deployment file so instead of running kubectl apply -f ./artifacts/operator-<arch>.yaml users will use kubectl apply -f ./artifacts/operator.yaml to deploy operator to cluster.
Checklist:
I have:

[x] updated the documentation and/or roadmap (if required)
[x] read the CONTRIBUTION guide
[x] signed-off my commits with git commit -s
[ ] added unit tests

This is a great PR, and thank you for doing the testing e2e. Let me know if you'd like to help me more with OpenFaaS and related projects. We have some areas that could benefit from this pattern.

I've created the build and the images on my laptop and pushed them up. I raised #14 to add CI

Yes, I would like to work on OpenFaaS. I just joined the slack :)
I can look at #14 and add travis.


data is not defined in screen.js line 338, it should be message.userid

@muaz-khan please also take a look at line 75 where screen.userid, where is variable "screen" come from?


Add 82599 T3 to the list of supported cards

The device id for the 82599 T3 card models is 0x151c, which was not added in the list of supported cards. The driver is working fine. We have a machine with two 82599 T3 cards. I could successfully deploy two VMs, launch snabb traffic and ping both machines.
link report:
  19 sent on A_NIC.tx -> A_Virtio.rx (loss rate: 0%)
  19 sent on A_Virtio.tx -> A_NIC.rx (loss rate: 0%)
  19 sent on B_NIC.tx -> B_Virtio.rx (loss rate: 0%)
  19 sent on B_Virtio.tx -> B_NIC.rx (loss rate: 0%)
load: time: 1.00s  fps: 3         fpGbps: 0.000 fpb: 0   bpp: 98   sleep: 100 us
load: time: 1.00s  fps: 4         fpGbps: 0.000 fpb: 0   bpp: 98   sleep: 100 us
load: time: 1.00s  fps: 4         fpGbps: 0.000 fpb: 0   bpp: 98   sleep: 100 us

Let me know if you want me to run more tests. What else should I try?

This is the result of the selftest for the driver:
sudo SNABB_TEST_INTEL10G_PCIDEVA="0000:04:00.0" \
    SNABB_TEST_INTEL10G_PCIDEVB="0000:04:00.1" \
    $SNABB_SRC/snabb snsh -t apps.intel.intel_app

-------
Send a bunch of packets from Am0
half of them go to nicAm1 and half go nowhere
link report:
                   0 sent on nicAm0.tx -> sink_ms.in1 (loss rate: 0%)
           3,742,877 sent on nicAm1.tx -> sink_ms.in2 (loss rate: 0%)
           7,486,290 sent on repeater_ms.output -> nicAm0.rx (loss rate: 0%)
                   2 sent on source_ms.out -> repeater_ms.input (loss rate: 0%)
-------
Transmitting bidirectionally between nicA and nicB
link report:
                   0 sent on nicA.tx -> sink.in1 (loss rate: 0%)
                   0 sent on nicB.tx -> sink.in2 (loss rate: 0%)
           3,874,215 sent on source1.out -> nicA.rx (loss rate: 0%)
           3,874,215 sent on source2.out -> nicB.rx (loss rate: 0%)
sq_sq: missing packets

@dpino That looks like a failed test to me. Packets are being sent but not received. Do you have the two ports cabled together? I believe this is expected by the current selftest function.
In your tests with VMs are you sending packets over the wire (physical cable) or are both VMs attached to the same NIC? If the latter then the test might work even if the NIC is not able to transmit/receive on the physical link.

You mean 82599 rather than 82559, right?

You should try to run the NFV selftest:
cd src; sudo TESTPCI=<pci_address> TELNET_PORT0=5000 TELNET_PORT1=5001 ./program/snabbnfv/selftest.sh`

This requires tmux, wget, make, telnet and numactl in the PATH.

@lukego When I run that test the NICs were cabled together 04:00.0 to 05:00.0. We changed it and cabled the ports of 04:00.0 together, but got the same result while testing the card.
Send a bunch of packets from Am0
half of them go to nicAm1 and half go nowhere
link report:
                   0 sent on nicAm0.tx -> sink_ms.in1 (loss rate: 0%)
           4,333,922 sent on nicAm1.tx -> sink_ms.in2 (loss rate: 0%)
           8,668,514 sent on repeater_ms.output -> nicAm0.rx (loss rate: 0%)
                   2 sent on source_ms.out -> repeater_ms.input (loss rate: 0%)
-------
Transmitting bidirectionally between nicA and nicB
link report:
                   0 sent on nicA.tx -> sink.in1 (loss rate: 0%)
                   0 sent on nicB.tx -> sink.in2 (loss rate: 0%)
           4,813,125 sent on source1.out -> nicA.rx (loss rate: 0%)
           4,813,125 sent on source2.out -> nicB.rx (loss rate: 0%)

@sleinen Right, fixed :)
@eugenia
Here is the result of running the SnabbNFV selftest.
sudo TESTPCI=0000:04:00.0 TELNET_PORT0=6060 TELNET_PORT1=6061 program/snabbnfv/selftest.sh

Waiting for VM listening on telnet port 6060 to get ready... [OK]
Waiting for VM listening on telnet port 6061 to get ready... [OK]
USING program/snabbnfv/test_fixtures/nfvconfig/test_functions/same_vlan.ports
1 packets transmitted, 1 received, 0% packet loss, time 0ms
PING succeded.
[  3]  0.0-10.0 sec  5.16 GBytes  4.43 Gbits/sec
IPERF succeded.
1 packets transmitted, 1 received, 0% packet loss, time 0ms
JUMBOPING succeded.
[  3]  0.0-10.0 sec  12.3 GBytes  10.6 Gbits/sec
IPERF succeded.
tx-checksumming: on
TX-CHECKSUMMING succeded.
tx-checksumming: on
TX-CHECKSUMMING succeded.
USING program/snabbnfv/test_fixtures/nfvconfig/test_functions/tx_rate_limit.ports
1 packets transmitted, 1 received, 0% packet loss, time 0ms
PING succeded.
IPERF (RATE_LIMITED) succeded.
IPERF rate is 814 Mbits/sec (900 Mbits/sec allowed).
RATE_LIMITED succeded.
1 packets transmitted, 1 received, 0% packet loss, time 0ms
JUMBOPING succeded.
IPERF (RATE_LIMITED) succeded.
IPERF rate is 814 Mbits/sec (900 Mbits/sec allowed).
RATE_LIMITED succeded.
USING program/snabbnfv/test_fixtures/nfvconfig/test_functions/rx_rate_limit.ports
1 packets transmitted, 1 received, 0% packet loss, time 0ms
PING succeded.
IPERF (RATE_LIMITED) succeded.
IPERF rate is 814 Mbits/sec (1200 Mbits/sec allowed).
RATE_LIMITED succeded.
1 packets transmitted, 1 received, 0% packet loss, time 0ms
JUMBOPING succeded.
IPERF (RATE_LIMITED) succeded.
IPERF rate is 814 Mbits/sec (1200 Mbits/sec allowed).
RATE_LIMITED succeded.
USING program/snabbnfv/test_fixtures/nfvconfig/test_functions/tunnel.ports
Resolved next-hop fe80::5054:ff:fe00:0 to 52:54:00:00:00:00
Resolved next-hop fe80::5054:ff:fe00:1 to 52:54:00:00:00:01
ND succeded.
1 packets transmitted, 1 received, 0% packet loss, time 0ms
PING succeded.
[  3]  0.0-10.0 sec  10.6 GBytes  9.07 Gbits/sec
IPERF succeded.
1 packets transmitted, 1 received, 0% packet loss, time 0ms
JUMBOPING succeded.
[  3]  0.0-10.0 sec  8.42 GBytes  7.23 Gbits/sec
IPERF succeded.
USING program/snabbnfv/test_fixtures/nfvconfig/test_functions/filter.ports
1 packets transmitted, 1 received, 0% packet loss, time 0ms
PING succeded.
Connection to fe80::5054:ff:fe00:0001%eth0 12345 port [tcp/*] succeeded!
PORTPROBE succeded.
Trying ::1...
Connected to localhost.
Escape character is '^]'.
nc -w 1 -q 1 -v  fe80::5054:ff:fe00:0001%eth0 12346
nc: connect to fe80::5054:ff:fe00:0001%eth0 port 12346 (tcp) timed out: Operation now in progress
root@vma:~#
root@vma:~# Connection closed by foreign host.
FILTER succeded.
USING program/snabbnfv/test_fixtures/nfvconfig/test_functions/stateful-filter.ports
1 packets transmitted, 1 received, 0% packet loss, time 0ms
PING succeded.
Connection to fe80::5054:ff:fe00:0001%eth0 12345 port [tcp/*] succeeded!
PORTPROBE succeded.
Trying ::1...
Connected to localhost.
Escape character is '^]'.
nc -w 1 -q 1 -v  fe80::5054:ff:fe00:0001%eth0 12348
nc: connect to fe80::5054:ff:fe00:0001%eth0 port 12348 (tcp) timed out: Operation now in progress
root@vma:~#
root@vma:~# Connection closed by foreign host.
FILTER succeded.
Trying ::1...
Connected to localhost.
Escape character is '^]'.
nc -w 1 -q 1 -v  fe80::5054:ff:fe00:0000%eth0 12340
nc: connect to fe80::5054:ff:fe00:0000%eth0 port 12340 (tcp) timed out: Operation now in progress
root@vma:~#
root@vma:~# Connection closed by foreign host.
FILTER succeded.

@dpino Success on the NFV selftest side! I am glad to see its easy to run the NFV selftest now (assuming it just ran out of the box for you).

So do I interpret correctly that:

NFV selftest passes and card is successfully passing traffic between locally attached VMs.
NIC selftest is failing and card is not successfuly passing traffic between physical ports.

In this case perhaps the most appropriate thing would be to add support but print a loud warning?

@lukego That's correct. I run the tests again to confirm it. I updated the PR printing out a warning in case device "0x151c" (82599 T3) is selected.

I think it would be better to have a more explicit error message saying that the T3 works for loopback traffic but not physical traffic. Then users will know what is expected to work and what is not. Could even print this in the intel10g device initialization perhaps?
Also: Please add commits to the PR instead of rebasing. I know, I have asked people to rebase in the past, but I am taking that back because I find that it makes Github based code review really hard and punishes people who merge PRs into their own trees for testing (which is a behavior we want to encourage).  See github-workflow.md for latest humble requests of contributors :).
We should also fix the driver. @javierguerragiraldez didn't you have this card working at some point? (IIRC it just needed a little time to initialize or something?)

I am reluctant to merge this. Based on Javier's comments it seems like we could add full support for the card without much more effort than marking it as broken/experimental.
@javierguerragiraldez Could we possibly persuade you to send a new PR that adds support for this card? Based on your comments it sounds to me like the NIC does work but the selftest needs to pause before testing for the link to be usable? However, there are a lot more code changes than this on your 10GbT branch so it would really help if you can strip this down to the minimum needed.

One idea would be that the selftest could have a "warm up" period. That is, if the link is not actually usable when it reports "link up" then the first step of selftest could be to loop until a packet is actually delivered (or timeout) before proceeding to the real tests. Reasonable?

@lukego OK, I will give it a try.

One idea would be that the selftest could have a "warm up" period. That is, if the link is not actually usable when it reports "link up" then the first step of selftest could be to loop until a packet is actually delivered (or timeout) before proceeding to the real tests.

I want to chime in and just suggest we document the eventual "workaround" really well. Like a big flashy comment plus explaining it in the commit message. I remember debugging NIC drivers as being not fun, and think we had issues before when our 10G driver code-rotted slightly. I am very fond of the strict intel selftest we currently have a exactly because of that. Maybe even even do a ifdef 82599 T3 dependent relaxation?

I've done some tests on the current next branch (which at the moment is the same as the just released 'mango') on Davos 89:00.0/.1 card:

snabb snsh -t apps.intel.intel_app passes without modification, but that's because the first tests are loopback, and take some time (100 restarts, 100 reconfigs...)  when it tries to send packets, the card seems to be already warmed up.

commenting out the three first tests makes the fourth one (sq_sq()) fail on 89:00 but it still passes on the SFP+ cards (i.e. 86:00)

simply adding a 2 sec delay between engine.configure(c) and engine.main({...}) makes it pass again.  1.5 sec seems not to be enough, 2.5 doesn't make it better.

thanks @javierguerragiraldez I tried the same steps on our machine with the T3 card and I got the following results (tested branch is 'next'):
Current state:
Send a bunch of packets from Am0
half of them go to nicAm1 and half go nowhere
link report:
                   0 sent on nicAm0.tx -> sink_ms.in1 (loss rate: 0%)
           4,284,632 sent on nicAm1.tx -> sink_ms.in2 (loss rate: 0%)
           8,569,785 sent on repeater_ms.output -> nicAm0.rx (loss rate: 0%)
                   2 sent on source_ms.out -> repeater_ms.input (loss rate: 0%)
-------
Transmitting bidirectionally between nicA and nicB
link report:
                   0 sent on nicA.tx -> sink.in1 (loss rate: 0%)
                   0 sent on nicB.tx -> sink.in2 (loss rate: 0%)
           4,811,340 sent on source1.out -> nicA.rx (loss rate: 0%)
           4,811,340 sent on source2.out -> nicB.rx (loss rate: 0%)
sq_sq: missing packets

sq_sq fails and mq_sq is not being executed. If I add a 2 second pause between sq_sq's configure and main:
Transmitting bidirectionally between nicA and nicB
link report:
           1,336,576 sent on nicA.tx -> sink.in1 (loss rate: 0%)
           1,336,576 sent on nicB.tx -> sink.in2 (loss rate: 0%)
           2,663,220 sent on source1.out -> nicA.rx (loss rate: 0%)
           2,663,220 sent on source2.out -> nicB.rx (loss rate: 0%)
-------
Send traffic from a nicA (SF) to nicB (two VFs)
The packets should arrive evenly split between the VFs
link report:
                   0 sent on nicAs.tx -> sink_ms.in1 (loss rate: 0%)
                   0 sent on nicBm0.tx -> sink_ms.in2 (loss rate: 0%)
                   0 sent on nicBm1.tx -> sink_ms.in3 (loss rate: 0%)
           8,574,885 sent on repeater_ms.output -> nicAs.rx (loss rate: 0%)
                   2 sent on source_ms.out -> repeater_ms.input (loss rate: 0%)
mq_sq: missing packets

sq_sq runs successfully but mq_sq fails. It seems mq_sq needs a 2 second pause too:
Send traffic from a nicA (SF) to nicB (two VFs)
The packets should arrive evenly split between the VFs
link report:
                   0 sent on nicAs.tx -> sink_ms.in1 (loss rate: 0%)
           1,731,322 sent on nicBm0.tx -> sink_ms.in2 (loss rate: 0%)
           1,731,364 sent on nicBm1.tx -> sink_ms.in3 (loss rate: 0%)
           3,463,410 sent on repeater_ms.output -> nicAs.rx (loss rate: 0%)
                   2 sent on source_ms.out -> repeater_ms.input (loss rate: 0%)
selftest: ok

sq_sq runs successfully but mq_sq fails. It seems mq_sq needs a 2 second pause too:

right.  it's because some of these tests not only reconfigure the app network but also reset the NIC.

I pushed a new commit that adds a time sleep before running tests 'sq_sq' and 'mq_sq'. The 2 second wait only runs if the card model is Intel 82599 T3.

@dpino I "did a Max" and sent a PR to your PR branch. Could you consider merging those changes please?
Generally this implementation seems good enough but I am still a little unclear. @javierguerragiraldez Is the T3 NIC reporting "link up" before it is able to send traffic? and another 2 seconds are needed? is the SNMP object that reports link status going to have a sane value? or is the issue that we are not trying hard enough to detect when the link is up before running the test?
The 2 second sleep seems fragile and likely to break in the future e.g. when testing other variants of the Intel NIC or with optical transceivers installed in the SFP+ NICs. This seems manageable though: it is only the selftest function that is affected and not the driver itself.
Generally I don't think it is part of the driver contract to be able to pass packets immediately upon startup (and in real life e.g. if we are cabled to a switch running Spanning Tree Protocol then it could take tens of seconds after link-up to actually be able to send traffic). So the selftest requirement of avoiding attempts to send before the link is ready is somewhat unusual.

@lukego: Sure, merged :)


Replace whitespace

I took @J-Fields  and addressed some issues in the PR in #770.
@structAnkit - I discovered j didn't work because it was injecting \r and \r\n makes XVim very confused. I modified it so it never injects \r anyway. Just \n which is the XVim convention. This solves 1 and 2 of your concern. Also, auto-indent works with a trick to do "interpret key events" against the source view.
@JugglerShu - test cases are added. hopefully this is good enough to merge now!

Fixes #767

@JugglerShu is there anything you'd like added in order to make this worthy of merging?