List directory content and copy files to and from images

My hst amiga library used for accessing Rigid Disk Blocks and formatting Fast File System and PFS3 partitions has recently been expanded with support for reading and writing files to and from both Fast File System and PFS3.
The idea is to add basic file system commands like dir, copy, rename, delete and move. Copy will support source set to a physical drive or image file and destination to a folder like d:\copy-to-amiga and vice-versa.
Example of listing partitions in Rigid Disk Block:
hst.imager.exe fs dir D:\test.vhd rdb

[20:32:59 INF] Hst Imager v0.1.0 (12/08/2022 15:31:48)
[20:32:59 INF] Henrik Noerfjand Stengaard
[20:32:59 INF] [CMD] fs dir D:\test.vhd rdb

Entries:

Name |  Size | Date | Attributes
-----|-------|------|-----------
DH0  | <DIR> |      |
DH1  | <DIR> |      |
DH2  | <DIR> |      |

3 directories, 0 files
[20:33:00 INF] Done

Example of listing Super Frog WHDLoad contents in Rigid Disk Block DH1 partition:
hst.imager.exe fs dir D:\test.vhd rdb/dh1/whdload/games/s/superfrog

[16:33:20 INF] Hst Imager v0.1.0 (12/08/2022 15:31:48)
[16:33:20 INF] Henrik Noerfjand Stengaard
[16:33:20 INF] [CMD] fs dir D:\test.vhd rdb/dh1/whdload/games/s/superfrog

Entries:

Name                 |      Size | Date                | Attributes
---------------------|-----------|---------------------|-----------
Disk.1               |    960 KB | 2006-09-08 06:47:30 | ----RWED
Disk.2               |    960 KB | 2006-09-08 06:48:00 | ----RWED
Disk.3               |    960 KB | 2006-09-08 06:48:30 | ----RWED
igame.iff            |   30.3 KB | 2021-09-29 09:30:28 | ----RWED
Manual               |   13.7 KB | 2001-05-03 09:22:50 | ----RWED
Manual.info          |      2 KB | 2021-07-29 04:08:26 | H---RWED
ReadMe               |      7 KB | 2021-07-29 04:07:18 | ----RWED
ReadMe.info          |      2 KB | 2021-07-29 04:08:26 | H---RWED
Superfrog.highs      | 160 bytes | 2001-05-03 09:51:30 | ----RWED
Superfrog.info       |   18.3 KB | 2021-07-29 04:08:30 | ----RW-D
Superfrog.slave      |    6.6 KB | 2021-07-29 04:07:20 | ----RWED
SuperfrogIntro.info  |   19.6 KB | 2021-07-29 04:08:36 | ----RW-D
SuperfrogIntro.slave |    3.4 KB | 2021-07-29 04:07:28 | ----RWED

0 directories, 13 files
[16:33:21 INF] Done

I'm making progress and have parts of it working. I can copy between directories, images (.img/.hdf/.vhd) and physical drives. I also added a extraction of .lha, .adf and .iso, which can extract to directory, image (.img/.hdf/.vhd) and physical drive.
It's currently in a testing phase stress testing FFS and PFS3 file systems fixing error occuring when reading or writing files.
For first version of this I'm limiting this functionality to following commands:

fs dir list directory in images/medias (.img/.hdf/.vhd,.lha,.adf,.iso) and physical drives.
fs copy copy files and directories from directories, images (.img/.hdf/.vhd) and physical drives to directories, images (.img/.hdf/.vhd) and physical drives.
fs extract extract files and directories from medias (.lha,.adf,.iso) to directories, images (.img/.hdf/.vhd) and physical drives.
adf create create blank adf file and optionally format it.

I'm not sure if it's worth making a prerelease as there are too many issues that needs fixing first with these new commands.

Most of the critical errors are fixed now and I have run a large stress test copying ~5.7GB whdload demos and games, which successfully copied all ~277000 files from a .hdf image file to a .vhd image file.
It took almost 12 hours and I noticed it was fast in the beginning and then over time became more slow (see attached screenshot of end result). So there is room for improving what ever is making it slower maybe with a profiler.
I also ran pfsdoctor experimental from pfs3aio v3.1 final forum thread at EAB and it reported 54 errors like:

ERROR:Anode #lx is double allocated
ERROR:Unknown extrafield flags found (ignored)
ERROR:Dangling rollover
ERROR:Corrupt directory entry

This is most likely related to how dirblocks are handled and how dir entries within dir blocks are managed when writing a lot of files to one or more directories. I have already done some fixes for this, so I have a good idea about what goes wrong. Fixing the errors requires creating a preferably small test case to reproduce the errors, so I can use those to examine and fix the errors.
For smaller tasks like extracting superfrog whdload .lha to a image file works fine and doesn't result in any errors when running pfsdoctor, so even with errors still present in pfs3 read and write functionality it feels like a good time to make a new prerelease.

I started fixing the errors discovered and solved a many of them, but still a few errors remain reported by pfsdoctor experimental:

2 corrupt dir entries.
1 reserved block nut used but allocated.
17 anodes not used but allocated.

Even with the errors, the image works and reads fine in WinUAE, so they are not super critical ones but still errors and therefore writing to PFS3 is still in an experimental phase. Reading seems to work fine.
I also did some fixes to caching used by PFS3 to ensure the cache gets flushed properly over time copying many files. The caching used linked list to store cached blocks read from disk. These have been replaced with dictionaries making cached block look up go from potentially have to go through each cached block in the lists to find out if the cache contains it to now have a directly look up using cached block no. or seq no.
This resulted in huge speed improvements and the same test as I did before when from ~12 hours to ~3 hours, so roughly 4 times faster for copying 5.7GB.

Lot of bugfixes and testing done over the last few months and it's now very stable. I haven't encountered any errors for the last couple of weeks and decided now is the time to release it.
v1.0.x is available for download now and it contains with following filesystem commands:

fs dir: list directory content in archives, disk images or physical drives.
fs copy: copy between local directories and disk images or physical drives.
fs extract: extract archives (.zip/.lha/.lzx/.z) to local directories, disk images or physical drives.

The remaining commands filesystem rename, delete, and move will be part of a future release.
Screenshots showing 5300+ whdloads being extracted from .lha and .lzx to a .vhd disk image, which took approx. 50 minutes for 6.4gb of files.

Screenshot showing same 6.4gb files and directories copied from one disk image to another, which took 1 hour and 6 minutes

I'm closing this issue now.


travis: Use no-shared for the FUZZING_BUILD_MODE_UNSAFE_FOR_PRODUCTION targets

There was actually a typo in it ...

+1 for the typo fix.

I added a new commit that should hopefully fix travis.

@richsalz: ping

+1,


Problem react-native 0.68.6 ios compile

No work in ios with multi tarjets
Multiple commands produce '/Users/pervasive/Library/Developer/Xcode/DerivedData/ramon-guecijowjmmrnafxmvkqpjpjlotr/Build/Products/Debug-iphonesimulator/freerasp-react-native/freerasp_react_native.framework/Headers/CryptoBridgingHeader.h'

Hello @gansu18,
Please try following:

Open your .xcworkspace file in XCode
Go to General > Frameworks, Libraries and Embedded Content
If there is Embed & Sign next to TalsecRuntime.xcframefork, change it to Do Not Embed

Let us know if that helps.
Additionally, we are very close to releasing version 2.0.0 of the FreeRASP package, which would resolve your problem with no actions needed from your side (other than importing the latest version of freeRASP). So maybe the easiest option now would be to wait for a couple of days until we release it, if you are willing to do so.
Best,
Tomas, Talsec developer

Hi @gansu18  ,
We just released v3.3.1, which fixes the issue, without having to remove use_frameworks!
Best,
Tomas, Talsec developer


1.15.2, omnisharp-1.30.1

Add the following to the changelog
## 1.15.2 (May 11, 1018)
* Fixed a 1.30.0 regression that prevented the script project system from working on Unix-based systems ([omnisharp-roslyn#1184](https://github.com/OmniSharp/omnisharp-roslyn/pull/1184), PR: [omnisharp-roslyn#1185](https://github.com/OmniSharp/omnisharp-roslyn/pull/1185)) _(Contributed by [@filipw](https://github.com/filipw))_

@DustinCampbell Good catch--thanks!

thanks! I just installed the VSIX from https://github.com/OmniSharp/omnisharp-vscode/releases/tag/v1.15.2-beta1 and it works like a charm ⭐


Bug: "build-script" folder in jenkins should exclude "smoketest jobs

What are you trying to do?
SmokeTests jobs should be excluded from "build-script" which is set in the regexp filter in the view/folder of jenkins
Expected behaviour:
All jenkins folder under "build-scripts" should only list build job, which is used as input for generatBuildMatrix.sh.
Observed behaviour:
e.g https://ci.adoptopenjdk.net/job/build-scripts/job/jobs/job/jdk8u/
should exclude all job matching pattern *SmokeTests
Any other comments:

duplicated as https://github.com/adoptium/ci-jenkins-pipelines/issues/282, this issue can be closed


ready.jstree called after only root nodes loaded

To my mind it shuld be after all nodes and childs. Now I use a workaround: setTimeout (...,100) to wait when all childs load.

Read the documentation - there are a few events you can use. My guess is you are using the state plugin - use the state_ready.jstree event.

I read the documentation. I init jstree in angular when resived data. tree is array of data
$(e).jstree({
"checkbox": {
"keep_selected_style": false,
"tie_selection": false
},
"search": {
"show_only_matches": true
},
"plugins": ["wholerow", "checkbox", "search"],
core: {
"themes": {
"icons": false
},
"data": tree
}
}).on('ready.jstree',
function() {
// I use setTimeout because on this event loaded only first level of tree. It's right?
setTimeout($.proxy(function () { scope.ngLoadModeldata(); }, this), 100);
}
);
Can you tell me it work right?

Sorry, I do not use angular, but ready.jstree fires after all nodes are loaded, there was a bug with it a few versions back, where it fired before all nodes were loaded, but that was fixed. Which version are you using?

I tested it on 3.1.0 and 2.8.x.
Do you speak russian?

What is 2.8.x? And no, sorry, I do not speak Russian.
As for your problem - let me clarify - tree is an array, right? If that is so the ready.jstree will fire immediately, what do you mean by "only root nodes loaded" - if you are passing an array, there is no lazy loading, which means all nodes are "loaded" immediately.
If you want to do something with the DOM, and you want to wait for nodes to open, that is another matter. If it is so - keep your timeout solution, it should even work with 0 timeout. But the DOM has nothing to do with jstree loading the nodes and firing the ready handler.
Best regards,
Ivan

Sorry, it was 3.0.8 version, not 2.8. thee os an array of countries whitch including towns as children elements. I wanna hide unselected items after load tree, but i shuld do timeout for test countries with some selected towns.
Thank you for converstation. May be in new version  will be added new event)

There will be no new event, the ready / loaded / init events are based on jstree parsing data, not on anything DOM related. Anyway, I am glad it is clear now.

I tested with 0 delay: it work throught time.

If I use lazy loading , i cannot programatically check_node , because open_node is take some time to render the  UI. in that case how i wait ?

There should be absolutely no problem checking a node - jstree does not depend on the DOM, so you can call check_node or select_node just with the node ID (not a selector) and all will work fine, regardless if the node is visible or not.
Best regards,
Ivan

Thank you for your fast reply. Actually what i'm doing is , dynamically opening nodes and going to check specific leaf nodes.
treex.jstree({
"plugins": ["checkbox"],
'core': {
'data': {
"animation": 0,
"check_callback": true,
"themes": {"stripes": true},
'url': function (node) {
return "/apis/" + node.id;
}
        }
    }
}).bind("loaded.jstree", function (event, data) {
    $.ajax({
        url: URL that will get my custom data, (selected nodes as paths)
        type: 'GET',
        async: false,
        headers: {
            Accept: "application/json"
        },
        success: function (response) {
                selectedTaxa = response.data;
            for (var i = 0; i < selectedTaxa.length; i++) {

                var ref = $('#data').jstree(true);

                try {
                    var nodeList = selectedTaxa[i].split("/");
                    for (var j = 0; j < nodeList.length; j++) {
                         //recursively open nodes . nodeList contain id of nodes
                         ref.open_node(nodeList[j]);
                    }
        // leaf node of the given path
         var nodeId = selectedTaxa[i].substring(selectedTaxa[i].lastIndexOf("/") + 1, selectedTaxa[i].length);
          ref.check_node(nodeId);
                } catch (ex) {

                }

            }

        },
        error: function () {
        }
    });

});

this will not get checked because of node is not rendered in the page. if we use same code after open all nodes , it will work. What is the problem here ?

Problem here is im loading the DOM dynamically and .

Hi Lahirue, I have the same problem. Did you solve it?

Same issue I'm facing as of @lahirue


Refactor: _Contract and ContractContainer

Overview
Provide a simple overview of what you wish to see added. Please include:

Remove partial _Contract creation and simplify creation logic

Specification

Remove partial() in the ape.__init__.py around _Contract creation
Rework _Contract and  get logic into ContractContainer class

Dependencies
Include links to any open issues that must be resolved before this feature can be implemented.

Outdated and Done


Post OTA_STOPPED_EVENT once new image verification finished

Post OTA_STOPPED_EVENT once new image verification finished
Description
After OTA job is finished and new firmware is verified successfully, there is no signal to coreMQTT-Agent network manager that an OTA job has stopped.
The signal is released only if OtaJobEventFail
Once case OtaJobEventStartTest there need to be posted: CoreMqttAgentManagerPost(CORE_MQTT_AGENT_OTA_STOPPED_EVENT); to recover other tasks.
Test Steps
All reproduction and test steps are described in the issue #73
Checklist:

[ x] I have tested my changes. No regression in existing tests.
[ ] I have modified and/or added unit-tests to cover the code changes in this Pull Request.

Related Issue
https://github.com/FreeRTOS/iot-reference-esp32c3/issues/73
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Thank you for creating this PR. We will look into it and discuss with you here.

I think this is the same issue here #7.


How to calibrate the camera?

I have successfully built this application on my iPad running iOS 10.3.3. Once the UI is open and I tap the Start button to calibrate nothing happens. I can see a message in Xcode saying calibration is starting but what to do next?

Hi! You need to print a chessboard pattern like this one and move it in front of the camera during the calibration step. You may find this video useful. Let me know if that helped!

Thanks for the reply. I tried with the checkerboard with AR markers inside it. Will let you know after I try with the checkerboard.


Bump kind versions + add comments about a footgun

This started out as a bump for kind versions and ended up in my delving into a massive footgun which caught me out relating to how we create CI kind clusters. Hopefully the comments explain that a little to try and avoid anyone else falling victim to this.
This is a follow-up to #4574
/kind cleanup
Bump kind image versions

/test pull-cert-manager-e2e-v1-21
/test pull-cert-manager-e2e-v1-20
/test pull-cert-manager-e2e-v1-19
/test pull-cert-manager-e2e-v1-18

The failure was expected, this happened in #4574 too. The gateway tests are broken by the new images, somehow.

The failures are coming from haproxy failing to start, e.g. in these logs

2021-11-09T18:17:49.631270691Z stderr F [ALERT] 312/181749 (33) : [haproxy.main()] Cannot raise FD limit to 4091, limit is 1024.

Interestingly kind also seems to be using haproxy internally https://github.com/kubernetes-sigs/kind/tree/main/images/haproxy, so I guess there must be a version of haproxy that works with this version of kind. Wonder if it'd be possible to use their haproxy build (random wondering, I've not looked into this at all)

Interestingly kind also seems to be using haproxy internally https://github.com/kubernetes-sigs/kind/tree/main/images/haproxy, so I guess there must be a version of haproxy that works with this version of kind. Wonder if it'd be possible to use their haproxy build (random wondering, I've not looked into this at all)

that is interesting, yeah! I couldn't really see anything obvious there, but there could be all kinds of subtle differences... I've ended up finding a happy solution I think, although this was more of an ordeal than I wanted it to be :joy:
hopefully the tests pass in CI like they did on my laptop...

/test pull-cert-manager-e2e-v1-21
/test pull-cert-manager-e2e-v1-20
/test pull-cert-manager-e2e-v1-19
/test pull-cert-manager-e2e-v1-18

/test pull-cert-manager-e2e-v1-19
please let the fail be a flake :upside_down_face:

/test pull-cert-manager-e2e-v1-19

I added a warning and a short sleep to make it more obvious. Given most tests which use a cluster take 35-45mins, a couple of  seconds shouldn't be noticeable in CI (but I can remove if people don't like the sleep - I think it helps make the warning more obvious on a dev laptop)


[maven-release-plugin] prepare release rocketmq-all-5.3.0

Which Issue(s) This PR Fixes

Fixes #issue_id
Brief Description

How Did You Test This Change?

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 43.98%. Comparing base (2bc0014) to head (47db160).

Additional details and impacted files
@@              Coverage Diff              @@
##             develop    #8379      +/-   ##
=============================================
- Coverage      43.99%   43.98%   -0.01%
- Complexity     10670    10672       +2
=============================================
  Files           1274     1274
  Lines          88939    88939
  Branches       11432    11432
=============================================
- Hits           39125    39120       -5
- Misses         44854    44860       +6
+ Partials        4960     4959       -1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Refactor create edit

This PR refactors 3 major areas in our application.

The audit template components for creating and editing
The organization components for creating and editing
The employee/profile settings components for creating and editing user profiles

The driving factor for this refactoring is that there are many areas in which there are code duplicates of fields, methods, and sometimes functionality. This is as a result of the HTML components that are either very similar (in the case of the employee/profile settings components) or that they use the same component.
The new structure is as follows:
The 2 previous components (for creating and editing) now will inherit similar functionality from a super class. Each of the components will also reuse the same HTML file, and the same SCSS file for displaying and styling the information.

Codecov Report

Merging #213 (78085c6) into master (33e495b) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #213   +/-   ##
=======================================
  Coverage   93.31%   93.31%
=======================================
  Files          35       35
  Lines         942      942
=======================================
  Hits          879      879
  Misses         63       63

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 33e495b...78085c6. Read the comment docs.

Codecov Report

Merging #213 (78085c6) into master (33e495b) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #213   +/-   ##
=======================================
  Coverage   93.31%   93.31%
=======================================
  Files          35       35
  Lines         942      942
=======================================
  Hits          879      879
  Misses         63       63

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 33e495b...78085c6. Read the comment docs.

Thank you for cleaning up the code with these refactorings!
Some notes:

For Organizations:

can you please adjust the Organizations table layout as the Status column's right padding squishes the Settings column

done

when I try to create a new organization and submit an empty name, the error "An organization with this name already exists"

fixed

For Employees:

When I try to create a new employee for an organization, the form allows me to submit a new user with the drop-down menu option set to "Choose a role".  The backend accepts the request and sets the newly created user's role to SA. This happens when I log in as an SA and IM.

I cannot replicate this, if you are talking about #212 , it is still a work in progress.

For Templates:

When I edit a template, the fields' + buttons are clickable. Please disable these until the Edit button is selected

Fixed

I noticed in manage-organizations\organization-list there is organization.dialog.html that I don't think it's being used anymore by the component. At the bottom of manage-organizations.components.ts, the class is there but appears to not hold much value. Can you confirm if we still need this dialog box, please?

Good catch, I remember looking at it and putting it on my mental to-do list, but forgetting!

Thank you for cleaning up the code with these refactorings!
Some notes:

For Organizations:

can you please adjust the Organizations table layout as the Status column's right padding squishes the Settings column

done

when I try to create a new organization and submit an empty name, the error "An organization with this name already exists"

fixed

For Employees:

When I try to create a new employee for an organization, the form allows me to submit a new user with the drop-down menu option set to "Choose a role".  The backend accepts the request and sets the newly created user's role to SA. This happens when I log in as an SA and IM.

I cannot replicate this, if you are talking about #212 , it is still a work in progress.

For Templates:

When I edit a template, the fields' + buttons are clickable. Please disable these until the Edit button is selected

Fixed

I noticed in manage-organizations\organization-list there is organization.dialog.html that I don't think it's being used anymore by the component. At the bottom of manage-organizations.components.ts, the class is there but appears to not hold much value. Can you confirm if we still need this dialog box, please?

Good catch, I remember looking at it and putting it on my mental to-do list, but forgetting!

For Employees:

When I try to create a new employee for an organization, the form allows me to submit a new user with the drop-down menu option set to "Choose a role".  The backend accepts the request and sets the newly created user's role to SA. This happens when I log in as an SA and IM.

I cannot replicate this, if you are talking about #212 , it is still a work in progress.

We ended discussing and @Amanda-Kolopa ended up showing me what she meant. Basically, it's that if no role was assigned, the front end would automatically assign the role to stock-keeper, and that the button to submit the form wasn't disabled when the form was invalid.

For Employees:

When I try to create a new employee for an organization, the form allows me to submit a new user with the drop-down menu option set to "Choose a role".  The backend accepts the request and sets the newly created user's role to SA. This happens when I log in as an SA and IM.

I cannot replicate this, if you are talking about #212 , it is still a work in progress.

We ended discussing and @Amanda-Kolopa ended up showing me what she meant. Basically, it's that if no role was assigned, the front end would automatically assign the role to stock-keeper, and that the button to submit the form wasn't disabled when the form was invalid.


MJPEG generate (useless) traffic when not used after preview

Home Assistant release (hass --version): 0.35.3
Python release (python3 --version): Python 3.4.2
Component/platform: Generic MJPEG IP Camera
Description of problem:
Might be (but not must be) connected with: #5235 @pvizeli
The configuration I use is webcam connected to a LEDE (OpenWRT) router with mjpg-streamer installed.
It's configured to stream HD / 30FPS at webport with username and password.
In HASS I can only configure like below:
"mjpeg_url: http://192.168.x.xx:xxxx/?action=stream"
Which makes HASS receiving stream even if no one is using it ang generating (WiFi thankfully) traffic.
mjpg-streamer only support 'action=stream' and 'action=snapshot' for MJPEG format.
Expected:
It would be superb to add 'mjpeg_snapshot_url' or something similiar to allow user declare an (optional) variable for "snapshot" preview and make HASS stream real "action=stream" only after opening the preview on webpage.
Providing 'action=snapshot' makes viewing stream unavailable and makes HASS show one, still image.
Problem-relevant configuration.yaml entries and steps to reproduce:
camera:
  - platform: mjpeg
    mjpeg_url: http://192.168.x.xx:xxxx/?action=snapshot
    name: Webcam
    username: xxx
    password: xxx

There's no problem with traffic after restarting the HASS - it only get snapshots.
But after looking (just once) for live-preview and closing HASS webpage, the stream continues.
Additional info:
on webpage: https://home-assistant.io/components/camera.mjpeg/ there's a blank bullet at the bottom - should there be something or it's just an mistake?

Snapshot/preview will at the moment generate with requests, so it will not affected with #5235  (aiohttp)

While we are at it. If a mjpeg stream is on the default "home tab" and also on an other tab. You can't really switch between them, because it will load it as seperate connection to the stream. If you switch back and fort on the tabs, can you actually reach "max allowed connection" on the IP camera itself.
Maybe just a coincidence...
Also not sure if its worth an other open issue page.

I think that https://github.com/home-assistant/home-assistant-polymer/pull/165 stops the stream now after closing the preview.

home-assistant/home-assistant-polymer#165 only fixes the problem caused by more-info view.
For snapshot cameras more-info turns the images into a stream.
If the camera is a stream camera to begin with, seems the stream should be manually stopped on tab switch and page unload.

This issue is still relevant. However the fix will probably be in the polymer repo.


Allow dot in parameter names for operations (#73)

The change is really trivial as I'm allowing dots in parameter names for operation. The diff is a bit bigger as I added some test as well as updated Swagger specs for Junit tests.
Fixes #73

Ping @macisamuele for review

Thanks .


Remove SGs from non-existent nodes ports on Update

Current code assumes that if a Node has been removed from the list on UpdateLoadBalancer(), we don't care about it's security group.
We could update applyNodeSecurityGroupIDForLB() to also lookup ports that have the SG tag and detach the SG from the ports that aren't belonging to the ports of the current list of Nodes.
https://github.com/kubernetes/cloud-provider-openstack/blob/04bd8e1343d630a0db3e9093dd71f59e477ee7ad/pkg/openstack/loadbalancer_sg.go#L72-L99

/cc dulek mdbooth
@dulek hey there, you filed a bug some time ago about this issue and I'm taking over it. If time permits, let me know what you think about the proposed fix.


Fix for widget not showing favorites when favicons are too large

Task/Issue URL: https://app.asana.com/0/414709148257752/1202381793018418
Tech Design URL:
CC:
Description:
Widget not showing favorites when favicons are too large. This fix:

Downsizes all existing large favicons to a max size of 192x192 as a one off migration
Downsizes large favicons before they are saved / cached

Steps to test this PR:

Install the app on simulator or device from last release and add twitter.com as a favorite. Add widget to your home screen and confirm the favicons are not loading in the widget. Also confirm in Debug > Image Cache that the bookmark cache image for twitter.com is 1024x1024
Install the app on the same simulator or device from this branch / PR and launch the app
Confirm in Debug > Image Cache that the bookmark cache image for twitter.com is now 192x192
Minimise the app and confirm that the widget now displays favicons
Visit si.com (which has a 512x512 favicon) in browser and add to favorites.
Confirm in Debug > Image Cache that its favicon size in both bookmarks and tabs caches is 192x192

Internal references:
Software Engineering Expectations
Technical Design Template
When ready for review, remember to post the PR in MM

Actually when running the app on develop I can't reproduce the part:
"Add widget to your home screen and confirm the favicons are not loading in the widget"
am I missing something?

tested on the device and it works, lgtm!


add prefix 'v' to version number

Currently the version number are of the X.Y.Z (e.g. 5.2.25, 6.0.0, 6.0.0-dev.1 etc.). However, it is good practice to use the prefix v, e.g. v5.2.25.
Implementation of this would require changes to the regexp in the following files:

[ ]  build.pl
[ ] publish.pl
[ ] build.m
[ ] easyspinupdate.m
[ ] easyspinversioncheck.m

and possibly more.

has been implemented and test


testkube-api: fix duplicate securityContext in MinIO

Pull request description
Checklist (choose whats happened)

[ ] breaking change! (describe)
[ ] tested locally
[ ] tested on cluster
[ ] added new dependencies
[ ] updated the docs
[ ] added a test

Breaking changes

Changes

Fixes

This should fix https://github.com/kubeshop/helm-charts/issues/431


Blog01 stanning

Hey guys, I was feeling nostalgic about the old days of battling with the baseline errors file, so I thought I'd nibble away at it a bit. In this PR I focus on the stanning errors emitted by the blog-01 example.

@spawnia great review, thanks for being awesome. Let me know if I missed anything.

Cool, thanks much for reviewing and helping out 👍


Crashplan upgrade

It looks like CrashPlan has an upgrade out.  To ensure I am using latest version of the image, I downloaded "latest", cleaned and restarted the container.  Code42  is waiting to retry an upgrade.

A quick update.  Clearing the browser cache made upgrade message go away.  I did however get a new version.  So view this as an reminder to download the latest version of the image and recreate the container, or in the case of Synology GUI, clear the container and start it up.

So just to mention that I pushed a new image with the latest CrashPlan version.

I have been pulling version "latest"  when downloading the image.  This morning I received CrashPlan 6.7.2.  I redid the process pulling your version v2.4.0.  This time I received CrashPlan 6.8.2.
Two items:

Should we be using "latest"?
I noticed the the three latest build on Docker Hub are showing a status "! Error".

The container does seem to be running normally.

There is currently an issue with the automated build system where pushing tags fails.  I had to manually push the latest tag.
You probably pulled the latest tag between the time v2.4.0 was available and the time I manually pushed the latest tag.
So yes, we should still use the latest tag.  It now properly points to v2.4.0.

So,  Upgraded with no problems.  Jlesage, thanks for making this work so well.  another support payment on the way.
you rock

So i noticed that paypal charged a $2.99 fee.  Do you have another way to make a donation?

Thanks for reporting.  Paypal is currently the only way but I will check if there are other alternatives.

how about venmo?
Regards,
Hal
On Thu, Aug 23, 2018 at 1:07 PM Jocelyn Le Sage<EMAIL_ADDRESS>wrote:

Thanks for reporting. Paypal is currently the only way but I will check if
there are other alternatives.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/jlesage/docker-crashplan-pro/issues/122#issuecomment-415495271,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AQcQKwctDpYRwbkQzj-omAqTs-O0GENVks5uTuFRgaJpZM4WGs_1
.

JL,
I think what surprised me about paypal was the wanted a $2.99 fee for a bank transfer.  Unlike credit cards, bank transfers cost pennies.

I really appreciate this image.  Thank you.  But, no offense, pushed images are inherently less trustworthy than automatically built ones.  Will you be able to go back to automatic builds?

Images are still built by the Docker Automated Builds system.  What make you think it’s no longer the case?

https://hub.docker.com/r/jlesage/crashplan-pro/builds/

If you are talking about the v2.4.0 build, you can look at the log to see that only the push of the latest tag has failed.  The image itself succeeded.  There was an issue at that time (https://github.com/docker/hub-feedback/issues/1611) that prevented the push of the tag, so I had to push it manually.  The tag being just an alias, you can see that both the latest and the built image have the save ID (hash): d43b112c9a8b.
I hope this answer your concern :)

Incase this helps anyone.  I've been using watchtower to rebuild crashplan image using whenever jlesage updates his repository/image.  It's a great tool - exports your crashplan config, updates the image and re-imports the config.
If this is the only docker image you're running following script will start watchtower looking for updates on 12 hour intervals
docker run -d  --name watchtower  -v /var/run/docker.sock:/var/run/docker.sock  v2tec/watchtower -i 43200 --cleanup
Full explanation and further switches to change schedule, restrict to named docker images or even local images:
https://github.com/v2tec/watchtower
If this cross-posting is against the rules-  sorry - please delete
Chris

Closing this issue.  Please re-open if needed.


Add versioning to misk.jobqueue.Job

Error Scenario
There is a race condition when deploying a new version of the application. The situation is:

A new version of the app will enqueue a job.
An already running (old version) of the app (job handler), which hasn't been killed by the deployment process yet, will attempt to process the event potentially causing issues. Assume that the job signature hasn't changed at all. The issue is more related with the business logic that the job handler implements.

Proposal: Adding versioning to misk.jobqueue.Job
A version in the Job (attributes) would help JobHandlers know if they are allowed to process a given Job. Otherwise, they would put the message back in the queue.
There are different things we can use as a message version (in order of preference):

Image creation timestamp: We would need the expose these somehow to the application.
Git Sha: consumers would have some awareness of the git repository to know if the SHA is included in the current deploy.
A manual version defined by application owners: Delegate this responsibility to application owners. However, if this is not automated is likely to be forgotten.

this seems specific to the schema of the job payload. why can't you add a version there?

We could do that. In fact, that's the last bullet of the proposal section in this issue. However, we were wondering if this could be somewhat automated; manually adding a version seems brittle specially when there is no change on the job payload structure, but rather how job handlers process the jobs.

If the structure hasn't changed and the old (possibly bugged) handler is capable of executing the job, broken business logic will eventually become fixed as the rollout completes. If the goal is to immediately avoid execution of any further broken logic, there are other levers we can use to halt consumption until new, fixed hosts are rolled out.
Of the three proposed options, I'd strongly lean on manual versioning — and at that point, I agree w/ Ryan; this can/should be app/handler specific.
Curious about the issues you encountered, would you mind elaborating?

Why do you lean on manual versioning? I see this option as unreliable because we need to update something to avoid this case manually, and it is likely to be forgotten.
Yes, there are other ways to avoid this scenario. We could have a different cluster for job handler, and making sure we deploy that cluster before producers.
We're trying to avoid needing to think about versioning on the consumer side to avoid error scenarios (even if they're transient).

Curious about the issues you encountered, would you mind elaborating?

evently = a service that produces events.
invest-flow = the service that consume events.
The data flow is as follows:
a. evently delivers an event
b. invest-flow creates a business object (an order, a customer, etc)
c. invest-flow enqueues a sqs job
d. invest-flow job handler picks the message and implement some business logic.
What happened was:

deploy started
pod1 subscribed to a new evently topic and it executed a.
pod1 received an evently event --so it executes b. and c.
pod2 executed d. --this is an old pod that has the old business logic (buggy business logic, or business logic that does not know how to handle the newly published job).

This error is a transient error because sqs job will retry, so hopefully at time of retry, pod2 is gone. It's worth noting that in this case the job payload has a type attr on it. pod2 only knows how to handle jobs of one type. pod1 knows how to handle jobs with two types.

Still not sure I understand the benefit of a version. Let's assume for the sake of argument that a version allows a Handler to decide if they can process an event. What does the handler do? It would just ignore the job and let another handler retry it later. How is that any different from what happened without a version? It's a more graceful error? While maybe that's a slight improvement to the error handling, you still have degraded job processing.
Personally as a developer I'd rather make my handlers backwards compatible so there is no disruption to my job handling processing. Old code can process new jobs for at least one release. I think we should encourage practices that increase job throughput.

The intent behind this proposal is to reduce the burden on application developers to build forward-compatible handlers.
The use of versioning seemed like a seamless way to handle this. However, we agree it has some negative impact on job throughput. Also, it could be harmful in rollback scenarios.
We also thought about changing our infrastructure split job handlers and producers in different clusters so that we can deploy them independently. However, it's not easy to draw a line to split them, especially in this case where the producer was an event handler.
Based on this analysis, we agreed that it's not good to hide this complexity from developers, and we should encourage best practices on forward-compatibility and increasing the job throughput.  So I'm closing this proposal.


Istio-proxy sidecars loose touch with istiod and don't get updates on endpoints

(NOTE: This is used to report product bugs:
To report a security vulnerability, please visit https://istio.io/about/security-vulnerabilities
To ask questions about how to use Istio, please visit https://discuss.istio.io)
Bug description
So we have a behavior that we see repeatedly in our cluster. The istio-proxy of some pods is loosing connectivity with the istiod and thus results to stale endpoints since the endpoints of different services have changed after re deployments etc.
Hitting the endpoint from inside the pod it gives us context deadline exceeded error. Seeing the output of istioctl pc endpoints <pod> it proves that the endpoint points to another IP of the actual target pod. Also seeing the output of istioctl ps the specific pod is not in the list.
Restarting pod solves the issue.
After enabling more verbose logging on istiod side and restarting things after a while a pod goes in this bad state and we can see in the istiod logs the following
2021-03-04 20:12:17 | 2021-03-04T19:12:17.504605Z	info	ads	Client closed connection sidecar~<IP_ADDRESS>~fuel-qagr-7c46c97458-rqzd8.mop~mop.svc.cluster.local-23
2021-03-04 20:12:17 | 2021-03-04T19:12:17.502646Z	info	ads	ADS: "<IP_ADDRESS>:46752" sidecar~<IP_ADDRESS>~fuel-qagr-7c46c97458-rqzd8.mop~mop.svc.cluster.local-23 terminated rpc error: code = Canceled desc = context canceled
2021-03-04 20:12:17 | 2021-03-04T19:12:17.502622Z	info	ads	Client closed connection sidecar~<IP_ADDRESS>~fuel-qagr-7c46c97458-rqzd8.mop~mop.svc.cluster.local-23
2021-03-04 18:17:25 | 2021-03-04T17:17:25.902706Z	warn	ads	type.googleapis.com/envoy.config.endpoint.v3.ClusterLoadAssignment: Send failure sidecar~<IP_ADDRESS>~fuel-qagr-7c46c97458-rqzd8.mop~mop.svc.cluster.local-23: rpc error: code = DeadlineExceeded desc = timeout sending Show context
2021-03-04 18:17:25 | 2021-03-04T17:17:25.902565Z	info	ads	Timeout writing sidecar~<IP_ADDRESS>~fuel-qagr-7c46c97458-rqzd8.mop~mop.svc.cluster.local-23
2021-03-04 18:17:20 | 2021-03-04T17:17:20.902467Z	info	ads	EDS: PUSH for node:fuel-qagr-7c46c97458-rqzd8.mop resources:1329 empty:0 cached:1328/1329
2021-03-04 18:17:20 | 2021-03-04T17:17:20.899122Z	info	ads	CDS: PUSH for node:fuel-qagr-7c46c97458-rqzd8.mop resources:1577

And nothing after that although specific pod is still up and running.
Now going inside the pod itself using tcpdump we can see packets coming and going to istiod.istio-system service on port 15012.
One detail is that the pod's port is not 46752 that we see in the above logs but something else.
Do you have any clue on how to debug this or what might be wrong? Is there any signs that we can see somewhere to understand further what's going on.
[ ] Docs
[ ] Installation
[x] Networking
[ ] Performance and Scalability
[ ] Extensions and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
[ ] Upgrade
Expected behavior
Steps to reproduce the bug
Not being able to reproduce this, it happens after some time
Version (include the output of istioctl version --remote and kubectl version --short and helm version --short if you used Helm)
istioctl output
client version: 1.8.2
control plane version: 1.8.3
data plane version: 1.8.3 (548 proxies)

kubectl output
Client Version: v1.18.10
Server Version: v1.19.7

How was Istio installed?
istioctl
Environment where the bug was observed (cloud vendor, OS, etc)
ubuntu
Additionally, please consider running istioctl bug-report and attach the generated cluster-state tarball to this issue.
Refer cluster state archive for more details.

46752

This is the local ephemeral port
When we hit timeout writing, we send an error back to the proxy, which is supposed to cause it to disconnect and reconnect. Given we timed out on the write, it seems like the pod may be in some bad state where its not receiving this error and not attempting to reconnect?

Now going inside the pod itself using tcpdump we can see packets coming and going to istiod.istio-system service on port 15012.

Is this just some keepalive packets or something, or giant XDS data (like megabytes of data)?

Hey @howardjohn, thanks for your reply.

Given we timed out on the write, it seems like the pod may be in some bad state where its not receiving this error and not attempting to reconnect?

is there a way to force re connection from the proxy side? can you think of any way or sings to debug this further to see why it gets to this bad state?

Is this just some keepalive packets or something, or giant XDS data (like megabytes of data)?

these are definitely small packets (either [P.] or just [.]in tcpdump, so no new connections are actually tried). Using-A` as output in tcpdump doesn't really help to see what's in there since it's probably grpc traffic. But it's not looking giant, more on the side of keepalive, but from both sides. I am wondering if it will be valuable to see traffic from the istiod side, although it's a bit more difficult since inside the istiod pods I don't have root. But if you think we could find something valuable I could definitely find a way to see traffic in there.

Is this just some keepalive packets or something, or giant XDS data (like megabytes of data)?

In case it helps I am adding couple packets here, which are mainly what we see. A lot 0 length packets (I guess keepalives) and the rest don't exceed 39 bytes.
07:12:10.927238 IP istiod.istio-system.svc.cluster.local.15012 > mypod.53774: Flags [P.], seq 1877:1916, ack 487, win 131, options [nop,nop,TS val<PHONE_NUMBER> ecr<PHONE_NUMBER>], length 39
E..[..@.@.,.dB.6dd.
:...8............\.....
...4........".1...[...../.1..@G.....9/...,.hBk.
07:12:10.927257 IP mypod.53774 > istiod.istio-system.svc.cluster.local.15012: Flags [.], ack 1916, win 136, options [nop,nop,TS val<PHONE_NUMBER> ecr<PHONE_NUMBER>], length 0
E..4(L<EMAIL_ADDRESS>dB.6..:.....8.......d......
...f...4


Cache file format

Input data is the only non-fixed-length field in the data. I should store the entirety of the data in fixed length format with indexes into a string array and a string table so I can read the entire database in with one command. I doubt very much that the data format will change very much in the future, and the speed improvement would by huge. Test by doing it without the input field first.

Not going to fix this right now.


Fix for webpack module federation "No version" error

fixes https://github.com/jashkenas/underscore/issues/2958

Coverage remained the same at 95.217% when pulling 46d77d3aabc897dd3e2fe069c5ffaacc1fd2ab02 on petschki:module-federation-version into da06656712e023ed40198310bab29634a7170ec2 on jashkenas:master.


Only send quota emails if confident that quota sizes are correct

We should only send out emails to users once our quota is being enforced and tests indicate that quota numbers are accurate.
HS <IP_ADDRESS>

likely this will be resolved by https://github.com/hydroshare/hydroshare/pull/5461

we are more confident now
resolved by Quota updates released as 2.14 move the HS datazone away from microservices.


OCPBUGS-14029: In UPI clusters VSphere platform could be nil

This PR resolves an issue when vsphere
platformSpec:
  type: VSphere
  vsphere: {}

is unset.

Adds a check for inf.Spec.PlatformSpec.VSphere, logs and returns if nil.

Adds a additional unit test to check this condition.

/jira refresh

/cherry-pick release-4.13

/lgtm
/approve


Toggl link missing in Capsule

Toggl link is not showing in Capsule anymore.

Tested, seems to be fixed with f9330200fd9d480149667a0703ce06ec93c0b837


Avoid conversion while getting values

Hello! I'm currently using this version of quick.db and there is a problem when getting values: for example I have the ID<PHONE_NUMBER>82813982 set to the key logchannel. Whenever I set it, it's all fine, but whenever I try and get the value it just gets converted to a number, and because of the JSON.parse precision limit it returns<PHONE_NUMBER>82814000. Looking at the source code, it looks like this is a problem also for normal quick.db. So I'm asking if the conversion can be avoided in any way.
EDIT: forgot to tell that json-bigint can be used as a substitute for the built-in JSON namespace.

You shouldn't be using such long numbers imo. Save as string and use it.

The problem is that whenever I try to save it, it saves as string. But whenever I get it, it converts to number.

I think I may have found a solution: just add a string of characters before the ID and it will get picked as a string overall. But this should be a temporary solution, it can't be like this. So if it's possible to save IDs as keys that would be amazing.

I doubt https://github.com/DevSnowflake/quick.db/blob/0aee816124a8ab99f8e3964ef8046076bebe4c12/src/Database.js#L137 line as the cause of parsing it twice (stringified data -> parsed data (string) -> parsed again (number)). Definitely needs to be fixed imo.