minor update on korean translation

Due to the difference in languages, there was some unnatural translation in Korean.
This will be gradually updated as found.

Hi @psychoet this PR have 22 commits, please can you create a new one with only the change you want to push or rebase it?

okay, I will create new branch and make new PR.


Cleanup the unwanted daemonsets mounts

We used to mount proc and certs from te host onto the agent containers
for some of the tools like prometheus metrics, we no longer need them
as we are using scale-ci-promdump and scraper for collection and analysis.

Fixes https://github.com/openshift-scale/workloads/issues/2


Fix parsing logic of function parameters with closing parentheses

Currently a function with parameter which have parentheses is wrongly parsed. As soon as a closing parentheses is encountered, the parsing logic assumes end state for the function parameter. For example, given  concat(sum(), 10) then sum( is treated as first parameter. This is wrong parsing logic.
A parameter should be treated completely parsed only when a , or, the last ) is encountered ( that is groupParen == 0)

@kallestenflo

@kallestenflo


Update monitor.md

Parameter changed healthcheck links here, but is not documented

🚀 Deployed on https://633c8ce8dacd921313840896--cozy-fox-0defec.netlify.app


[next-react-query-tailwind]: Fix Milestone for Issues and pull requests tab for single repo

Currently milestones filter doesn't work by targeting the ID of the milestone.
Apparently, we can't filter by milestones for pull requests with the Graphql API so, you will need to extend the API by using the REST API version for that situation
Fix

replace milestone with milestoneNumber in the query
replace the id of the milestone with the number of the milestone

I added some functions for the app rest API
restAPI.ts has the fetch API and the APIs needed to label, milestones and get the pull requests.
I also added the pull request parser named parseRESTAPIPullRequest to format the data to the format used to display the PR's
You can look at the Qwik and SolidStart showcase as a guide.
Please ensure you pull the latest from the main branch to see the changes in the Qwik or SolidStart
@Megio


Closing sidenav with escape does not work when mode is "side"

Reproduction
Steps to reproduce:

Open this stackblitz
Click the sidenav.toggle() button to open sidenav
Press escape on your keyboard

Expected Behavior
What behavior were you expecting to see?
I was expecting the sidenav to close.
Actual Behavior
Sidenav does not close.
Environment

Angular: 8.2
CDK/Material: 8.2
Browser(s): Chrome
Operating System (e.g. Windows, macOS, Ubuntu): macOS

Workaround
There seems to be 2 workarounds. Related SO question.

Setting focus inside the sidenav, and then pressing escape
Adding a @HostListener on the esc key

We decided against fixing this, because the fix would depend on how the sidenav is used in the application. See the discussion in https://github.com/angular/components/pull/17967. For your case you should be able to fix it by adding adding another keyboard listener and using the API to close it.


Service Register

Hi, i want to register my own service. I cannot find any function to achieve this. Can anyone please explain me how to register my mdns service? Thanks in advance.

This is literally on the second line in their Readme, are you sure you searched enough? Beside, why opening the issue here?


Content controller test cases

Merging even though the Sonar gate is failed because the files added/modified are related to test cases


Add short sentence describing what "ACL" means

It's not common terminology and for those unaware it isn't immediately obvious what ACL stands for. I added a short sentence just for simple clarification.

Thanks for your input!


Album Cover

I'm using your Apple Music/iTunes rich presence program for 1-2 years now and I really appreciate your work!
I would like to have the album art displaying instead of the Apple Music icon. You could set the Apple Music icon as the small image and the album art as the large one.
Keep up the good work!

Closing due to iTunesRichPresence no longer being maintained.


When scrolling down the navbar at top hides automatically

Issue: #200
When scrolling down the navbar at top should hide automatically

@Aashutoshbro Thanks for tackling, but tests are failing :(


Namespace issue

I have a collection named "Route". After a lot of debugging "TypeError: object is not a function" I realized my collection was being used by Restivus in place of its internal Route class. I already check, your not exporting Route and it is not available in global scope. Yet by changing three lines of your code I made "Route" -> "Restivus.Route". This fixed my issue, likely a global scope problem.
I can continue to consume your package locally, version locked to 0.8.3, but wondered if you had any suggestions.

Sorry about that. Not sure why that's happening when those variables shouldn't be exposed to the global scope. I'm happy to accept an update for this. Do you want to submit a pull request with the necessary changes? Please just read the guidelines before submitting.

Will do, how do you want it Namespaced? I did Restivus.Route for ease but anything would work really.

Hmmm. Good question. I don't want to expose it outside the package if I can avoid it. Restivus is exported, so my first thought is that we shouldn't put it there. Gimme a sec. It's been a while since I've actually looked at the Restivus code :P

See https://github.com/Kestanous/meteor-restivus if you want to see what I did, still reviewing your guidelines so it may need some work.

After some thought I think it may be a coffeescript issue. I think package top level this "@" is actually app global. I recall having an issue with this a long time ago. I believe that is one of the reasons I now only write packages in js vs coffee, and keep coffee for apps only.

I was just looking into that. I think you're correct. But the meteor docs suggest there may be some fixes easier than converting the entire package to js. From the meteor docs on CoffeeScript:

There is no way to make a package-scope variable from a .coffee file other than exporting it. We couldn't figure out a way to make this fit naturally inside the CoffeeScript language. If you want to use package-scope variables with CoffeeScript, one way is to make a short .js file that declares all of your package-scope variables. They can then be used and assigned to from .coffee files.

If you want to share variables between .coffee files in the same package, and don't want to separately declare them in a .js file, we have an experimental feature that you may like. An object called share is visible in CoffeeScript code and is shared across all .coffee files in the same package. So, you can write share.foo for a value that is shared between all CoffeeScript code in a package, but doesn't escape that package.

They've been claiming that feature to be experimental for over a year now, and they still haven't removed it from the docs, so folks must have not complained about it being broken. We could try that as well. I think I prefer the first option though. Not a huge fan of having this random share variable floating around code. Any preference? Do you mind creating a lib/package-scope-variables.js file and testing this approach? Feel free to name it something different if you think of anything less verbose.

Hmm yeah, I will give the first a shot. It would be nice to solve this.

Awesome. Keep me posted. Thanks for following up on this!

So as far as I can tell option one does not work, we would have to define the classes in the package-scope-variables.js file. Which we can't do if you want to use Coffeescript classes. Defining the class as one of those vars in another file is still file scoped, Route = {} will be true for all other files. Maybe I am doing it wrong but I don't think so.
Option two could work as an alternative but at that point it is the same as my solution (and I think mine is cleaner).
At this stage I need to ask why. Why don't you want the Route class to be available externally, though hidden under some scope (e.g. Restivus).

Restivus isn't a namespace right now. It's a class. It feels weird to me to nest one class under another like that.

Fair point, though IMO an object is an object. We could use the magic shard object instead. It is semantics really, so whatever your comfortable with.
One thing to note is your tests prove Route is on the Global scope as they can access it directly. If we fix this you are going to need a way for your tests to find that object. I proved that when testing option one. I think we can add the lib/route.coffee in the onTest function as a fix but I am not sure. All around this scoping is a bit of a headache.

The other option is to introduce another "namespace" that will behave exactly like Route does now, and may interfere in other projects that are currently using Restivus.  So I'd rather work with the share object. I just tested it locally and it works as expected, without any need to change the API. You just have to add the file in the test definition as well to get the tests to pass, which is certainly less than ideal, but OK for now. I'll have to reconsider how the tests are setup at some point, but that's not any worse than what's happening now (with the global namespace being polluted). I agree that this is an extremely annoying issue with CoffeeScript in Meteor packages.

I just published v0.8.5 with the fix for this. Let me know if you have any issues with that.

Thank you, sorry lost track of this. We are in production now with my quick fix so it may be a while before I get to testing this in the wild.


Cotede config format

At the moment, running the CoTeDe tests produces a huge number of warnings:
Deprecated cfg format. It should contain a threshold item.

This is due to a change in config file format in CoTeDe, which this PR attempts to address. The json changes in cotede_qc/qc_cfg/ are meant to reflect the pattern and spirit of https://github.com/castelao/CoTeDe/commit/23dfdca96da7942514d4563da3fa26f23d5620a5 and https://github.com/castelao/CoTeDe/commit/3819361becc7669542aadc0892ae2d653ad06089, and have the desired effect of eliminating the warnings - but as I have not had much involvement in the CoTeDe integration part of this project, I'd like a second opinion before we merge; cc @castelao @s-good.

This certainly looks reasonable to me. These config files were taken from CoTeDe and then trimmed to remove the bits not needed for AutoQC, so it is correct that any changes to the CoTeDe config files should be matched in AutoQC versions.

I've just been looking at #173 and have realised that the code that runs the CoTeDe tests has changed considerably since I last worked on it. It looks like these config files may now be not needed. The warnings are probably being triggered by the fuzzy logic config file since this config is now called fuzzylogic in CoTeDe instead of fuzzy. However, #173 reveals that there is an issue that things are not working quite right. I will reply to that issue to explain.

As commented in #173 and #176 the CoTeDe copies of the config files were not being used due to a missing import and other problems. This was fixed in #175. Once #176 is complete the AutoQC copies of the config files will no longer be needed. The warning messages described above should not appear once https://github.com/castelao/CoTeDe/pull/29 is merged.

Sounds right to me - let's close this PR out in favor of the stack of changes you cite above.


Problem with particles.js 2023

On iOS devices the animation is not displayed correctly. Has anyone been able to use the library is it iOS 16? In my case, the problem is in the devices of that version

The last commit was in 2017, this repository was abandoned since then. You can try tsParticles instead


Dbainstanceparameter

changed object and string to Dbainstanceparameter for source, destination and sqlinstance

merged! we're looking so good. thanks for the kick in the pants, shawn!


Rename package from pinecone-client to pinecone

Problem
We want to rename this package from pinecone-client to pinecone.
Solution

Update pyproject.toml to refer to the new name pinecone
Find and replace references to old package name in README and docstrings
Remove nightly publish job that would need to be updated but has been broken for a long time for other reasons

After this rename, we will need to setup a separate project to wrap this and continue publishing it under the old pinecone-client name for people who may not know about the rename, very similar to how old releases have been republished as pinecone using this wrapper project.
Type of Change

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update
[ ] Infrastructure change (CI configs, etc)
[ ] Non-code change (docs, etc)
[x] None of the above: We plan to publish a wrapper package under the name pinecone-client. So this shouldn't be a breaking change for end users consuming updates via the old name.

Test Plan
Describe specific steps for validating this change.

What's the reason for renaming the package?

What's the reason for renaming the package?

We went through a long process this year to get control of the package name pinecone which was previously being squatted by an inactive project. Pinecone is our trademark, so makes sense that we should have the name since it wasn't really being used. Now we have to do some work to migrate over everything (docs, etc) but don't want to break compatibility with anything else out there referencing the old name.


[tool_crash] ProcessException: Process exited abnormally:xcodebuild: error: Scheme Runner is not currently configured for the build action. Command: /usr/bin/env, OS error code: 66

Command
flutter build ios

Steps to Reproduce

...
...
...

Logs
ProcessException: Process exited abnormally:
xcodebuild: error: Scheme Runner is not currently configured for the build action. Command: /usr/bin/env, OS error code: 66
#0      RunResult.throwException (package:flutter_tools/src/base/process.dart:172:5)
#1      _DefaultProcessUtils.run (package:flutter_tools/src/base/process.dart:392:21)
<asynchronous suspension>
#2      buildXcodeProject (package:flutter_tools/src/ios/mac.dart:362:66)
#3      _rootRunUnary (dart:async/zone.dart:1198:47)
#4      _CustomZone.runUnary (dart:async/zone.dart:1100:19)
#5      _FutureListener.handleValue (dart:async/future_impl.dart:143:18)
#6      Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:696:45)
#7      Future._propagateToListeners (dart:async/future_impl.dart:725:32)
#8      Future._completeWithValue (dart:async/future_impl.dart:529:5)
#9      _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:40:15)
#10     _completeOnAsyncReturn (dart:async-patch/async_patch.dart:311:13)
#11     _runBuildWithRetries (package:flutter_tools/src/ios/mac.dart)
#12     _rootRunUnary (dart:async/zone.dart:1198:47)
#13     _CustomZone.runUnary (dart:async/zone.dart:1100:19)
#14     _FutureListener.handleValue (dart:async/future_impl.dart:143:18)
#15     Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:696:45)
#16     Future._propagateToListeners (dart:async/future_impl.dart:725:32)
#17     Future._completeWithValue (dart:async/future_impl.dart:529:5)
#18     _AsyncAwaitCompleter.complete (dart:async-patch/async_patch.dart:40:15)
#19     _completeOnAsyncReturn (dart:async-patch/async_patch.dart:311:13)
#20     _DefaultProcessUtils.run (package:flutter_tools/src/base/process.dart)
#21     _rootRunUnary (dart:async/zone.dart:1198:47)
#22     _CustomZone.runUnary (dart:async/zone.dart:1100:19)
#23     _FutureListener.handleValue (dart:async/future_impl.dart:143:18)

[✓] Flutter (Channel stable, 1.22.3, on Mac OS X 10.15.7 19H2, locale en-GB)
    • Flutter version 1.22.3 at /Users/apple/Developer/flutter
    • Framework revision 8874f21e79 (6 days ago), 2020-10-29 14:14:35 -0700
    • Engine revision a1440ca392
    • Dart version 2.10.3

[✓] Android toolchain - develop for Android devices (Android SDK version 29.0.2)
    • Android SDK at /Users/apple/Library/Android/sdk
    • Platform android-29, build-tools 29.0.2
    • ANDROID_HOME = /Users/apple/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 12.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.1, Build version 12A7403
    • CocoaPods version 1.9.3

[✓] Android Studio (version 3.5)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 44.0.1
    • Dart plugin version 191.8593
    • Java version OpenJDK Runtime Environment (build 1.8.0_202-release-1483-b49-5587405)

[✓] VS Code (version 1.50.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.16.0

[!] Connected device
    ! No devices available

! Doctor found issues in 1 category.

Flutter Application Metadata
Type: app
Version: 1.0.7+7
Material: true
Android X: false
Module: false
Plugin: false
Android package: null
iOS bundle identifier: null
Creation channel: master
Creation framework version: 068fa84c5bb28382853a0506395158afa30f3cc6
Plugins
firebase_messaging-6.0.16
flutter_plugin_android_lifecycle-1.0.8
geocoder-0.2.1
geolocator-5.3.2+2
google_api_availability-2.0.4
google_maps_flutter-0.5.30
location_permissions-3.0.0+1
path_provider_linux-0.0.1+2
shared_preferences-0.5.8
shared_preferences_linux-0.0.2+1
shared_preferences_macos-0.0.1+10
shared_preferences_web-0.1.2+7
url_launcher-5.7.10
url_launcher_linux-0.0.1+1
url_launcher_macos-0.0.1+7
url_launcher_web-0.1.5
url_launcher_windows-0.0.1+3

What's the output of:
$ cd path/to/project/ios
$ xcrun xcodebuild -configuration Debug -workspace Runner.xcworkspace -list

Open ios/Runner.xcworkspace in Xcode
Product > Scheme > Managed Schemes
Select Runner > Edit...

In the Build tab are any Targets selected?
Do you see this little checkmark?

See also https://github.com/flutter/flutter/issues/47261#issuecomment-567202724

Without additional information, we are unfortunately not sure how to resolve this issue.
We are therefore reluctantly going to close this bug for now.
Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.
Could everyone who still has this problem please file a new issue with the exact description of what happens, logs, and the output of flutter doctor -v.
All system setups can be slightly different, so it's always better to open new issues and reference related issues.

Without additional information, we are unfortunately not sure how to resolve this issue.
We are therefore reluctantly going to close this bug for now.
Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.
Could everyone who still has this problem please file a new issue with the exact description of what happens, logs, and the output of flutter doctor -v.
All system setups can be slightly different, so it's always better to open new issues and reference related issues.


\text fails with \ref, \eqref

Issue Summary
\text fails with \ref and \eqref.
Technical details:
Latest MathJax 3, Firefox.
Supporting information:
<!DOCTYPE html>

<meta charset="UTF-8" />

First section

\begin{equation}
y=mx+b \label {eq-thisequation}
\end{equation}

Text macros

eqref in math text: \(\text{\eqref {eq-thisequation}}\)

eqref in math: \(\eqref {eq-thisequation}\)

ref in math text: \(\text{\ref {eq-thisequation}}\)

ref in math: \(\ref {eq-thisequation}\)

Thanks for pointing this out.  These should be made active in the textmacros.
In the meantime, you can use \(\text{$\ref{eq-thisequation}$}\) as a workaround.


Add host fns for concatenating strings

A couple times I've seen folks ask how to concatenate Strings. Strings in Soroban are just Bytes and their type is just a name to change how tooling might present those bytes.
Related Discord discussion: https://discord.com/channels/897514728459468821/1162161366854013039/1171851834751721472

Similar issue https://github.com/stellar/rs-soroban-sdk/issues/1280
We might be solving it on the sdk side by leveraging bytes_append host function and the String<-> Bytes interop.

+1 as requested in the developer survey

Cross-posting from https://github.com/stellar/rs-soroban-sdk/issues/1280:

@jayz22 How can String convert to and from Bytes? I don't see any existing SDK functionality for converting between them, and I don't see any host functions for doing so either.


Add "value has.duplicate" to type=address and type=service filters

Is your feature request related to a problem?
Some customers have very large configurations with tens of thousands of address and service objects. In these large environments, the likelihood of objects with duplicate values rises significantly. It is important to be able to identify them.
Describe the solution you'd like
I would like to be able to query very large configurations to find all objects that have the same value, such as address objects, address range objects, address groups (containing the same members), and FQDNs. There only needs to be one instance of <IP_ADDRESS>/8 in shared or TCP_22 in shared :)
Describe alternatives you've considered
Expedition can do this, but Expedition has a GUI and it can be buggy and takes a long time to load large numbers of objects. This tool can do it on an offline config file or interact with a device's API.
Additional context
No additional context regarding the problem.

PAN-OS-PHP has for duplicate values a different entry point;
to display duplicate values, which are really possible to merge, this is the actual approach:

use offline config file or download:
pan-os-php type=upload in=api://[MGMT-IP] out=config.xml

address
pan-os-php type=address-merger in=config.xml out=/dev/null location=any allowmergingwithupperlevel | tee address-merger-log.txt
pan-os-php type=addressgroup-merger in=config.xml out=/dev/null location=any allowmergingwithupperlevel | tee addressgroup-merger-log.txt

service
pan-os-php type=service-merger in=config.xml out=/dev/null location=any allowmergingwithupperlevel | tee service-merger-log.txt
pan-os-php type=servicegroup-merger in=config.xml out=/dev/null location=any allowmergingwithupperlevel | tee service-merger-log.txt

your FR is in combination with actions=exportToExcel a next step.
Do you think with the actual available functionality this is enough for you to continue?
Please always think about, a plain filter can only display that an object with same name / same value is available.
the merger scripts are displaying this in a specific run method, and already can solve this with a single run.

another option to display this is using the argument "exportCSV"
pan-os-php type=address-merger in=config.xml out=merged-config.xml location=any allowmergingwithupperlevel exportCSV | tee address-merger-log.txt
at the end of the script a full list is displayed, which object can be merged with another one

another additional option is to display all the set commands for the merger change:
pan-os-php type=address-merger in=config.xml out=merged-config.xml location=any allowmergingwithupperlevel exportCSV=exportCSV.html outputformatset=setcommands.txt | tee address-merger-log.txt

for the above, please update to the newest version

Regarding using type=address-merge to accomplish this task: address-merge automatically replaces and deletes the duplicate objects with no input or direction from the user. You can use a different output file to preserve the input file, but if we want to use pan-os-php in an auditing fashion this method isn't as elegant as the "actions + filters" format of the standalone rule, address, and service scripts. This also is not desirable if you are using in=api://production_device instead of in=offline_config.xml.
Also the automatic replacement feature may be not desirable for some customers. They may want to choose the address object to replace the duplicates with -- for example, a Panorama config has 10+ objects with different names but the same value of <IP_ADDRESS>/8. These objects could be called "DANS_RFC1918_SUBNET", "net_<IP_ADDRESS>_8", "rfc_1918_10-space", etc.   The address-merge script may choose "DANS_RFC1918_SUBNET" instead of the more descriptive "net_<IP_ADDRESS>_8" name. If there is a way to choose the "object to replace" I do not know how to do that. I could not find much documentation for the address-merge script.
I'm interested in using the address script as an auditing tool to look through the customer's config and listing all the address objects with duplicate values to show them what they have, give them time to review a clean report such as the exportToExcel option in the other php utils, and allow them to be more specific with what objects to replace duplicates with.

if you like to choose a object with a specific object name use the pickfilter argument:
'pickfilter=(name regex /^net/)'
If you like to use the merger script as an audit tool, I recommend right now to use the offline config method.

download config
pan-os-php type=upload in=api://MGMT-IP out=offlineConfig.xml

run merger against offline config
pan-os-php type=address-merger in=offineConfig.xml out=/dev/null allowmergingwithupperlevel exportCSV=mergedLog.html location=any 'pickfilter=(name regex /^net_/) or (name regex /^h-/)'

All information about what could be merged is available in the HTML file defined in argument exportCSV.
If you are running the merger script against in=api://MGMT-IP but you do not like to merge, but display,
you will not get the full capacity out of what can be merged, please keep in mind that the script is going up from lowest child to shared DG, and calculate after each single DG merge the whole possibilities for the next DG completely new.

I rechecked some of you commands:
please validate the new argument available at the develop Container:
docker pull swaschkut/pan-os-php:develop
docker run -v ${PWD}:/share -it swaschkut/pan-os-php:develop
the new argument actions=display:
pan-os-php type=address-merger in=api://MGMT-IP location=any allowmergingwithupperlevel exportcsv=merger-log.html actions=display

can you please check if the already available filter;
type=address/service
- overriden.at.lower.level
- overrides.upper.level
is giving you the expected output?
pan-os-php type=address in=api://MGMT-IP 'filter=(object override.at.lower.level)'
or
pan-os-php type=address in=api://MGMT-IP 'filter=(object overrides.upper.level)'

the new functionality is implemented and is not changing anything on your environment:
pan-os-php type=address-merger actions=display in=api://MGMT-IP allowmergingwithupperlevel exportCSV=mergedLog.html location=any 'pickfilter=(name regex /^net_/) or (name regex /^h-/)'
of course this is also available for:
type=service-merger
type=tag-merger
type=addressgroup-merger
type=servicegroup-merger
Please validate, and if needed reopen this FR

Hi Sven,
Sorry I missed this over the weekend. I'll test it out today.
Thanks!
On Mon, Apr 25, 2022 at 5:18 AM Sven Waschkut @.***>
wrote:

the new functionality is implemented and is not changing anything on your
environment:
pan-os-php type=address-merger actions=display in=api://MGMT-IP
allowmergingwithupperlevel exportCSV=mergedLog.html location=any
'pickfilter=(name regex /^net_/) or (name regex /^h-/)'
of course this is also available for:
type=service-merger
type=tag-merger
type=addressgroup-merger
type=servicegroup-merger
Please validate, and if needed reopen this FR
—
Reply to this email directly, view it on GitHub
https://urldefense.com/v3/__https://github.com/PaloAltoNetworks/pan-os-php/issues/534*issuecomment-1108307438__;Iw!!Mt_FR42WkD9csi9Y!ciSHEMDEuT_eImpzhi16Nf7cJhaojAu7ZYlNWCkiSLYI9b9b3TW_WQf9NfbWpIblzBjURROsvrA26hgrplhrRqDiPOpQUXU$,
or unsubscribe
https://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AVHL3VQXVXT2SQVZ6WF2K5DVGZPOHANCNFSM5RU7A2ZA__;!!Mt_FR42WkD9csi9Y!ciSHEMDEuT_eImpzhi16Nf7cJhaojAu7ZYlNWCkiSLYI9b9b3TW_WQf9NfbWpIblzBjURROsvrA26hgrplhrRqDi3PPp73M$
.
You are receiving this because you authored the thread.Message ID:
@.***>

--
Tom Addair
Sr. Professional Services Consultant
Palo Alto Networks


Add new membership attacks

Description
Added two membership inference attacks: one rule-based and one learned black-box attack.
Added two Jupyter notebooks explaining how to apply both membership and attribute inference attacks.
Added dependency on pandas for preprocessing of nursery dataset used in new notebooks.
Type of change

[ ] Bug fix (non-breaking)
[x] New feature (non-breaking)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[x] This change requires a documentation update

Testing
Added tests for new membership attacks in tests/attacks/inference/test_membership_inference.py
Test Configuration:

OS: MacOS
Python version: 3.7
ART version or commit number
TensorFlow 2.1 / Keras 2.3.1 / PyTorch 0.1

Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

Codecov Report

Merging #538 into dev_1.4.0 will decrease coverage by 1.08%.
The diff coverage is 14.50%.

@@              Coverage Diff              @@
##           dev_1.4.0     #538      +/-   ##
=============================================
- Coverage      66.03%   64.94%   -1.09%
=============================================
  Files            135      136       +1
  Lines          12106    12359     +253
  Branches        2096     2143      +47
=============================================
+ Hits            7994     8027      +33
- Misses          3396     3617     +221
+ Partials         716      715       -1

Impacted Files
Coverage Δ

art/attacks/inference/membership_inference.py
0.00% <0.00%> (ø)

art/estimators/classification/tensorflow.py
49.30% <12.50%> (-1.41%)
:arrow_down:

art/estimators/classification/keras.py
71.02% <21.87%> (-5.45%)
:arrow_down:

art/estimators/classification/pytorch.py
61.27% <32.35%> (-3.54%)
:arrow_down:

art/defences/detector/evasion/detector.py
69.79% <100.00%> (+0.64%)
:arrow_up:

...fences/detector/evasion/subsetscanning/detector.py
81.08% <100.00%> (+0.17%)
:arrow_up:

...tors/certification/randomized_smoothing/pytorch.py
47.27% <100.00%> (+0.97%)
:arrow_up:

...s/certification/randomized_smoothing/tensorflow.py
34.69% <100.00%> (+1.36%)
:arrow_up:

...t/estimators/classification/detector_classifier.py
90.19% <100.00%> (+0.09%)
:arrow_up:

art/estimators/classification/ensemble.py
66.23% <100.00%> (+0.44%)
:arrow_up:

... and 10 more

Opened a new pull request from a clean branch (573)


Add arguments used to start Robot Framework to the test report, to give insight in provided arguments that produced the results

When test reports are shared among team members, it can be quite useful if the report itself contains the commandline arguments that were used to start Robot Framework.
The use cases I see are:

debugging
referencing older test reports compared to a recent one

The most important values I want to be able to see in the report, are the values passed to:
--argumentfile
--include
--exclude
--variable
Is there already a way I could (programmatically) grab all the arguments provided to Robot Framework and add them to the log myself?
If not, I think this is an enhancement regarding traceability.

there already is variable named ${OPTIONS} https://github.com/robotframework/robotframework/issues/5201

Indeed. Unfortunately, it does not yet give/contain all the information that I am looking for.
But, thanks for the link. I see I can add feature requests for enhancements op OPTIONS in that topic. Thanks!


aurchain: solving versioned dependencies

With #5 restricting to official repositories, this could actually work. How complex would it be to implement?
See also https://github.com/rmarquis/pacaur/commit/d3f7d552da4eebdb6be5279124f1a9b0906039bc

One option would be to call pacsift on the given arguments, before continuing with the rest of the pipeline. However, this means a second pacsift call on every run for something that only happens marginally.

On a side note, beware that it is sometimes possible to have a package both in the official repositories and in the AUR. As an example, some perl subpackages provided by the main perl package can have a newer version in the AUR. For that scenario, you'd have to check the version of the requested package too, not only the name.

You can do this already by combining aursift and aursearch; I'll leave checking versions to the user.

I mean, some package can depend on a specific version of a package that is both provided in the repo and the AUR. If you don't do any version check, the dependency solver can fail.
Granted, it's not a common situation, but that these are still valid PKGBUILDs. I believe I've seen perl and haskell packages with that user case.

If you could link me some of those problematic packages, that would be great. :+1:

Unfortunately, my old user case isn't valid anymore since Perl has had many update since then (see first few lines of https://github.com/falconindy/cower/issues/91 for details). This report is for cower, but since then I moved the dependency solver fully on the bash side in pacaur.
I've had a look at the packages provided by the current perl package, but only a handful are also in the AUR with a newer version. None of them are required by another AUR package, so that issue does currently not happen with perl. I'll see if I find some user case with Haskell packages, but Haskell on Arch is kinda a nightmare anyway.

Haven't found any Haskell package either, so you can ignore that issue for now. I'll ping you if I find such package again.

The -U switch to aursift should handle this transparently, by comparing sync versions with the AUR. If it doesn't, please mention so.

@rmarquis you mentioned a similar example here: https://bbs.archlinux.org/viewtopic.php?pid=1688551#p1688551
I personally think those cases are stupid and maintainers should just like, maintain their packages, but it's probably not great to find in the middle of a long build that makepkg fails on some version check.
Some things I thought about:

don't strip version information but leave it in a third column
when removing duplicates, use pacsort on the list and remove duplicates keeping the highest version
check the AUR if said highest version is available.

I don't know if that covers all cases, e.g. if for some reason there's a package bar that has depends foo<9001.0' and the aur has foo 9000.0`, the build would succeed but the above would say it wouldn't.

I personally think those cases are stupid and maintainers should just like, maintain their packages, but it's probably not great to find in the middle of a long build that makepkg fails on some version check.

I agree. In most case these versioned deps are even completely unnecessary, but in practice they are quite annoying since they sometimes prevent builds to succeed.
Pacaur does complete versioning check, but this considerably slow down the dependency solver. Haven't found a better implementation yet.

The issue would be trivial to solve if the AUR supported querying versioned pkgnames, e.g. foo>=42.0.
https://bugs.archlinux.org/task/54906

Pacaur does complete versioning check, but this considerably slow down the dependency solver. Haven't found a better implementation yet.

One of the newcomers aurman seems to have solved this while keeping good performance: https://github.com/polygamma/aurman/issues/17#issuecomment-370155538

while keeping good performance

This is done by loading everything into memory and thus avoiding all i/o bottleneck.

This is done by loading everything into memory and thus avoiding all i/o bottleneck.

That is indeed true, but there is more than that.
Let me explain the whole aurman dependency solving in more detail... Guess that's interesting for a few people. I will fill the explanations with code snippets from the aurman source code.

Two expac queries yield the names of all installed packages and the names of all known repository packages.

repo_packages_names = set(expac("-S", ('n',), ()))
installed_packages_names = set(expac("-Q", ('n',), ()))

Hence I am able to classify:
installed_repo_packages_names = installed_packages_names & repo_packages_names
unclassified_installed_names = installed_packages_names - installed_repo_packages_names

3 expac queries and 1 aur rpc query yield all needed information:
# installed repo packages
if installed_repo_packages_names:
    return_list.extend(
        Package.get_packages_from_expac("-Q", list(installed_repo_packages_names), PossibleTypes.REPO_PACKAGE))

# installed aur packages
installed_aur_packages_names = set(
    [package.name for package in Package.get_packages_from_aur(list(unclassified_installed_names))])

if installed_aur_packages_names:
    return_list.extend(
        Package.get_packages_from_expac("-Q", list(installed_aur_packages_names), PossibleTypes.AUR_PACKAGE))

unclassified_installed_names -= installed_aur_packages_names

# installed not repo not aur packages
if unclassified_installed_names:
    return_list.extend(Package.get_packages_from_expac("-Q", list(unclassified_installed_names),
                                                       PossibleTypes.PACKAGE_NOT_REPO_NOT_AUR))

In general the following things are received and saved for packages:
self.name = name  # %n
self.version = version  # %v
self.depends = depends  # %D
self.conflicts = conflicts  # %H
self.optdepends = optdepends  # %o
self.provides = provides  # %P
self.replaces = replaces  # %R
self.pkgbase = pkgbase  # %e
self.install_reason = install_reason  # %w (only with -Q)
self.makedepends = makedepends  # aur only
self.checkdepends = checkdepends  # aur only
self.type_of = type_of  # PossibleTypes Enum value
self.repo = repo  # %r (only useful for upstream repo packages)
self.groups = groups  # %G

Now for the upstream packages:
1 expac query receives all known upstream packages from repositories.

Package.get_packages_from_expac("-S", (), PossibleTypes.REPO_PACKAGE)

Easy, isn't it? Now for the upstream aur packages:
In the beginning i am collection "names". "names" the user put on the command line + names of known (installed) aur packages. I want to receive information about packages just once, so I fetch everything I might possibly need at once. No fetching is needed during dependency solving. The algo for that is quite short and intuitive:
packages_names = set([strip_versioning_from_name(name) for name in packages_names])
packages_names_to_fetch = [name for name in packages_names if name not in self.all_packages_dict]

while packages_names_to_fetch:
    fetched_packages = Package.get_packages_from_aur(packages_names_to_fetch)
    self.append_packages(fetched_packages)

    deps_of_the_fetched_packages = []
    for package in fetched_packages:
        deps_of_the_fetched_packages.extend(package.relevant_deps())

    relevant_deps = list(set([strip_versioning_from_name(dep) for dep in deps_of_the_fetched_packages]))

    packages_names_to_fetch = [dep for dep in relevant_deps if dep not in self.all_packages_dict]

Notice: Since I request everything I might possibly need, aurman is able to solve the dependency chain of ros-indigo-desktop-full since "the missing dep" is listed elsewhere, and hence aurman knows of the existence.
That's it. All needed information about packages, great.
Now for the interesting part: Datastructures for the performance.
The main concept I came up with is the concept of a "System". A "System" is nothing more than a collection of packages, but if packages "belong together" you may calculate many things before you actually need them. A system consists of:
self.all_packages_dict = {}  # names as keys and packages as values
self.repo_packages_list = []  # list containing the repo packages
self.aur_packages_list = []  # list containing the aur but not devel packages
self.devel_packages_list = []  # list containing the aur devel packages
self.not_repo_not_aur_packages_list = []  # list containing the packages that are neither repo nor aur packages

# reverse dict for finding providings. names of providings as keys and providing packages as values in lists
self.provides_dict = {}
# same for conflicts
self.conflicts_dict = {}

Every time a package is being added to a system, it is classified as repo, aur but not devel, devel or nor aur nor repo package. Also are the relevant entries added for the reverse dicts. Notice: Appending packages does not delete any of those information, hence you do not recalculate anything of that, if you only append. Code for the appending to the reverse dicts, since that may be interesting:
def __append_to_x_dict(self, packages: Sequence['Package'], dict_name: str):
    dict_to_append_to = getattr(self, "{}_dict".format(dict_name))

    for package in packages:
        relevant_package_values = getattr(package, dict_name)

        for relevant_value in relevant_package_values:
            value_name = strip_versioning_from_name(relevant_value)
            if value_name in dict_to_append_to:
                dict_to_append_to[value_name].append(package)
            else:
                dict_to_append_to[value_name] = [package]

called like self.__append_to_x_dict(packages, 'provides')
Checking if e.g. all deps of a package are fulfilled is hence done quite easily:
for dep in package.relevant_deps(only_make_check=only_make_check, only_depends=only_depends):
    if not self.provided_by(dep):
        return False
else:
    return True

self.provided_by(dep) is just searching the reverse dict which is very performant.
Notice: Differentiating between depends, make_depends and check_depends is no problem since we differentiated when receiving the packages.
Before we get to the dep solving algorithm itself, one more important thing:
Hypothetical appending of packages to a system. It allows to calculate what happens when installing packages without having to install them. Seems like a trivial idea, which it is, but this really is one of the most important things of aurman. The algo for that works like this ((german) english first, then python):
Split the packages to append into chunks: AUR packages are installed one by one, repo packages together. E.g. you want to install Repo1, AUR1, Repo2, Repo3, Repo4, AUR2 in that order:
The 4 chunks will be: Repo1, AUR1, Repo2 AND Repo3 AND Repo4, AUR2
For every chunk do the following:

Check if packages in chunk conflict each other (really only relevant for chunks containing more than one package). If they conflict, do not append, next chunk.
Calculate all conflicting packages of the system to append to. Includes packages with same names, they will be "upgraded" but technically removed first. Not to have special cases keeps the code sane.
Remove the conflicting packages and append to packages to append.
Now check, if there are packages in the system, whose deps are not fulfilled anymore. Iterative and done until there are no such packages.

Python code:
new_system = System(list(self.all_packages_dict.values()))
if not packages:
    return new_system

chunked_packages = System.calc_install_chunks(packages)
last_index = len(chunked_packages) - 1

for i, package_chunk in enumerate(chunked_packages):
    # check if packages in chunk conflict each other
    package_chunk_system = System(())
    for package in package_chunk:
        if package_chunk_system.conflicting_with(package):
            break
        package_chunk_system.append_packages((package,))
    # no conflicts
    else:
        # calculate conflicting packages
        conflicting_new_system_packages = []
        for package in package_chunk:
            # print why packages will be removed
            if packages_names_print_reason is not None:
                will_be_deleted = new_system.conflicting_with(package)
                for package_to_be_removed in will_be_deleted:
                    if package_to_be_removed.name in packages_names_print_reason:
                        aurman_note(
                            "Package {} will be removed due to a conflict with {}".format(
                                Colors.BOLD(Colors.LIGHT_MAGENTA(package_to_be_removed.name)),
                                Colors.BOLD(Colors.LIGHT_MAGENTA(package.name))))

            conflicting_new_system_packages.extend(new_system.conflicting_with(package))
        conflicting_new_system_packages = set(conflicting_new_system_packages)

        # remove conflicting packages
        if conflicting_new_system_packages:
            deleted_packages = True
            if print_way:
                print("   {} : {}"
                      "".format(Colors.BOLD(Colors.LIGHT_RED("Delete"))
                                , ", ".join([Colors.BOLD(Colors.LIGHT_MAGENTA(package))
                                             for package in conflicting_new_system_packages])))
            for package in conflicting_new_system_packages:
                del new_system.all_packages_dict[package.name]
            new_system = System(list(new_system.all_packages_dict.values()))
        else:
            deleted_packages = False

        # append packages
        if print_way:
            print("   {}: {}"
                  "".format(Colors.BOLD(Colors.LIGHT_GREEN("Install"))
                            , ", ".join([Colors.BOLD(Colors.LIGHT_MAGENTA(package))
                                         for package in package_chunk])))
        new_system.append_packages(package_chunk)

        # last exit brooklyn
        # final check for sanity of the whole solution
        # we do not accept mistakes!
        if not deleted_packages and not (i == last_index):
            continue

        # delete packages whose deps are not fulfilled anymore
        while True:
            to_delete_packages = []
            for package in new_system.all_packages_dict.values():
                if packages_names_print_reason is not None and package.name in packages_names_print_reason:
                    if not new_system.are_all_deps_fulfilled(package, only_depends=True, print_reason=True):
                        to_delete_packages.append(package)
                else:
                    if not new_system.are_all_deps_fulfilled(package, only_depends=True):
                        to_delete_packages.append(package)

            if not to_delete_packages:
                break

            if print_way:
                print("   {} : {}"
                      "".format(Colors.BOLD(Colors.LIGHT_RED("Delete"))
                                , ", ".join([Colors.BOLD(Colors.LIGHT_MAGENTA(package))
                                             for package in to_delete_packages])))
            for package in to_delete_packages:
                del new_system.all_packages_dict[package.name]
            new_system = System(list(new_system.all_packages_dict.values()))

return new_system

Notice: last exit Brooklyn should not be needed, as stated in the comment, but there may be cases where the installed system of the user is in an inconsistent state and although nothing gets deleted, there are still packages without all deps fulfilled. so: better safe than sorry... @rmarquis ;)
Finally: The dependency solving...
When are you able to say, that there really is no possible way of finding a solution to a dependency problem? A really important observation is: Since there are depends, make- and checkdepends, you HAVE to differentiate between them, to be able to tell if there is no solution. Since I am a mathematician at heart I tried to solve a simpler problem first... So, forget about check and make. Everything is a normal dependency.
Another problem: "Normal" graph theory does not provide a useful representation (as far as I know), because there may be multiple providers for dependencies.
But another thing first: What are dependencies?
Hence dependencies are names. Strings. Nothing more. It may be the case, that a dependency is being provided by a package with the name of the dependency itself, great, but we do not really care.
Let us simplify more. There are no problems in our simplified world. No dep-cycles, no conflicts, deps which are not providable. This is simple enough to state the following:
You want to install packages A, B, C, ...
The dependencies of the packages are A_a, A_b, A_c, ...., B_a, B_b, ... (no duplicates), (no deps of deps)
The number of possible providers for the dep X_x is denoted as |X_x|.
Hence there are |A_a| * |A_b| * ... * |B_a| * ... solutions for our dep problem.
Now let's go back, step by step. Back to the real world...
That there may be deps of deps and duplicates is not a real problem, since we will take a recursive approach and track for which deps we already chose a provider.
So, there are "real problems" now. We do not differentiate between the different types of problems as of now. A valid solution for the dependency problem consists of an ordered list of packages. You install the packages in order and you are done. No chunking yet (nevertheless only needed because of dep-cycles in repos like between mesa and libglvnd - damnit)
Let us assume you have a partial solution consisting of two packages A and B.
You want to append C and a problem occurs. We do not know anything about problems with concrete packages, there are just problems. What can be done? Well, this is pretty straight forward.
All dependencies need to be fulfilled for the solution to be valid, simple boolean algebra AND.
A single dependency needs to be provided by at least one provider, so simple boolean OR between the providers. So e.g. we have all in all the deps X, Y and Z. X is being provided by A, A_1, Y by B, B_1 and Z by C, C_1. Hence we get:
(A OR A_1) AND (B OR B_1) AND (C OR C_1)
We already tried A, B, C which yields a problem, hence we can try the finite number of other possibilites.
This is fine, but impracticable. Let's say we know between which packages the problem persists.
E.g. we know the problem is between B and C. We can conclude that trying A_1 instead of A will not change anything, hence we are able to reduce the number of other solutions to try. Seems trivial, but this is an important oberservation.
Back to dependencies of dependencies. Now this trivial observation becomes mighty. For every package in the partial solution is clear, why it is in the solution. The package has been chosen as a provider for a dependency. Hence we are able to track the way of decisions which lead to this exact package being in the partial solution. Hence we are able to conclude which dependencies are worth to try other providers for. If a dependency is not included in the decision chain which lead to the problem packages being in the solution, changing anything there will in fact not change anything about the problem. In practice this reduces the number of other solutions to try drastically.
Simplest case: You got a package, which depends on something for which no provider exists. The only way to solve this problem is, to track the path backwards, to check, if there is a possibility not wanting to install this package.
Second case: Conflict. You check for all conflicting packages the decision trees to see if you are able to decide so that no conflict occurs.
Third and to be honest the coolest case, since it's about one package but maybe two decision trees:
Dep cycle. When finding the dep-cycle, it is possible that you reached the dep on different paths. E.g.
A -> B -> C -> D -> E -> C
The paths are A -> B -> C and D -> E -> C
Funny to track recursively, but possible and the solution is quite nice
Notice: Hence it is indeed possible to solve dep-cycles... If it is for example possible to provide E by F and F depends on nothing, we found a valid solution.
Great, we have almost covered everything, just one more thing. Deps, make- and checkdeps.
This really required quite a bit of thinking, well, the other things too, but this one... meh...
The solution is: Get rid of the make- and check depends. That means: Assume you only want to install a single package. If there is no solution for that, fine, then we are done. If there is, we are able to BUILD that package and hence do not have to worry about make- and check depends. Do that for every package. When you have done that, guess what? Only depends matter... Now the algo in python:
Have commented everything, so you will be surely finding the puzzle pieces I have discussed in the python code.
Calling of the algo
@staticmethod
def dep_solving(packages: Sequence['Package'], installed_system: 'System', upstream_system: 'System') -> List[
    List['Package']]:
    """
    Solves deps for packages.

    :param packages:                The packages in a sequence
    :param installed_system:        The system containing the installed packages
    :param upstream_system:         The system containing the known upstream packages
    :return:                        A list containing the solutions.
                                    Every inner list contains the packages for the solution topologically sorted
    """

    deps_to_deep_check = set()
    single_first = False

    while True:
        current_solutions = [DepAlgoSolution([], [], set())]
        found_problems = set()

        # calc solutions
        # for every single package first
        if single_first:
            for package in packages:
                new_solutions = []
                for solution in current_solutions:
                    solution.dict_call_as_needed = {package.name: True}
                    new_solutions.extend(
                        package.solutions_for_dep_problem(solution, found_problems, installed_system,
                                                          upstream_system, deps_to_deep_check))
                current_solutions = new_solutions

        # now for all packages together
        for solution in current_solutions:
            solution.dict_call_as_needed = {}
            for package in packages:
                solution.dict_call_as_needed[package.name] = True
        for package in packages:
            new_solutions = []
            for solution in current_solutions:
                new_solutions.extend(
                    package.solutions_for_dep_problem(solution, found_problems, installed_system, upstream_system,
                                                      deps_to_deep_check))
            current_solutions = new_solutions

        # delete invalid solutions
        current_solutions = [solution for solution in current_solutions if solution.is_valid]

        # in case of at least one solution, we are done
        if current_solutions:
            break

        deps_to_deep_check_length = len(deps_to_deep_check)
        for problem in found_problems:
            problem_packages_names = set([package.name for package in problem.relevant_packages])
            deps_to_deep_check |= problem_packages_names

        # if there are no new deps to deep check, we are done, too
        if len(deps_to_deep_check) == deps_to_deep_check_length and single_first:
            break
        elif len(deps_to_deep_check) == deps_to_deep_check_length:
            if len(packages) > 1:
                single_first = True
            else:
                break

    # output for user
    if found_problems and not current_solutions:
        aurman_error("While searching for solutions the following errors occurred:\n"
                     "{}\n".format("\n".join([aurman_note(problem, False, False) for problem in found_problems])),
                     True)

    return [solution.packages_in_solution for solution in current_solutions]

Algo itself
def solutions_for_dep_problem(self, solution: 'DepAlgoSolution', found_problems: Set['DepAlgoFoundProblems'],
                              installed_system: 'System', upstream_system: 'System',
                              deps_to_deep_check: Set[str]) -> List['DepAlgoSolution']:
    """
    Heart of this AUR helper. Algorithm for dependency solving.
    Also checks for conflicts, dep-cycles and topologically sorts the solutions.

    :param solution:                The current solution
    :param found_problems:          A set containing found problems while searching for solutions
    :param installed_system:        The currently installed system
    :param upstream_system:         The system containing the known upstream packages
    :param deps_to_deep_check:      Set containing deps to check all possible dep providers of
    :return:                        The found solutions
    """

    def filter_solutions(solutions: Sequence['DepAlgoSolution']) -> List['DepAlgoSolution']:
        """
        Filter given solutions so that only valid solutions are left
        or in case of no valid solutions only one invalid solution

        :param solutions:   The solutions to filter
        :return:            The filtered solutions
        """
        return_solutions: List['DepAlgoSolution'] = []

        for solution in solutions:
            if not return_solutions:
                return_solutions.append(solution)
                continue

            first_solution = return_solutions[0]
            if first_solution.is_valid and solution.is_valid:
                return_solutions.append(solution)
            elif first_solution.is_valid:
                continue
            elif solution.is_valid:
                return_solutions = [solution]

        return return_solutions

    if self in solution.installed_solution_packages:
        return [solution.solution_copy()]

    # dep cycle
    # dirty... thanks to dep cycle between mesa and libglvnd
    if self in solution.visited_packages and not (self.type_of is PossibleTypes.REPO_PACKAGE):
        # problem only relevant
        # if the solution is not already invalid
        if solution.is_valid:
            index_of_self = solution.visited_packages.index(self)
            cycle_packages = []
            for i in range(index_of_self, len(solution.visited_packages)):
                cycle_packages.append(solution.visited_packages[i])
            cycle_packages.append(self)

            # create the problem
            cycle_problem = DepAlgoCycle(cycle_packages)
            for package in cycle_packages:
                cycle_problem.relevant_packages.add(package)
                cycle_problem.relevant_packages |= set(solution.dict_to_way.get(package.name, []))
            found_problems.add(cycle_problem)
        invalid_sol = solution.solution_copy()
        invalid_sol.is_valid = False
        return [invalid_sol]

    # pacman has to handle dep cycles between repo packages
    elif self in solution.visited_packages:
        return [solution.solution_copy()]

    # copy solution and add self to visited packages
    solution: 'DepAlgoSolution' = solution.solution_copy()
    is_build_available: bool = self in solution.packages_in_solution
    own_way: List['Package'] = solution.dict_to_way.get(self.name, [])
    own_not_to_delete_deps: Set[str] = set()
    solution.visited_packages.append(self)
    current_solutions: List['DepAlgoSolution'] = [solution]

    # filter not fulfillable deps
    relevant_deps = self.relevant_deps()
    for dep in relevant_deps[:]:

        # skip since already provided
        if installed_system.provided_by(dep):
            continue

        # skip since built package available and dep is not a normal dependency
        # so it's make and/or check dep
        if is_build_available and dep not in self.relevant_deps(only_depends=True):
            continue

        # dep not fulfillable, solutions not valid
        if not upstream_system.provided_by(dep):
            for solution in current_solutions:
                solution.is_valid = False

            # create problem
            dep_problem = DepAlgoNotProvided(dep, self)
            dep_problem.relevant_packages.add(self)
            dep_problem.relevant_packages |= set(own_way)
            found_problems.add(dep_problem)

            relevant_deps.remove(dep)

    # AND - every dep has to be fulfilled
    # we filtered the unfulfillable deps,
    # hence at least one dep provider is available
    for dep in relevant_deps:

        # skip since already provided
        if installed_system.provided_by(dep):
            continue

        # skip since built package available and dep is not a normal dependency
        # so it's make and/or check dep
        if is_build_available and dep not in self.relevant_deps(only_depends=True):
            continue

        # fetch dep providers
        dep_providers = upstream_system.provided_by(dep)
        dep_providers_names = [package.name for package in dep_providers]
        dep_stripped_name = strip_versioning_from_name(dep)

        # we only need relevant dep providers
        # deps_to_deep_check will be filled
        # when we encounter problems as dep-cycle, conflicts ...
        if dep_stripped_name in dep_providers_names and dep not in deps_to_deep_check:
            dep_providers = [package for package in dep_providers if package.name == dep_stripped_name]

        # OR - at least one of the dep providers needs to provide the dep
        finished_solutions = [solution for solution in current_solutions if dep in solution.visited_names]
        not_finished_solutions = [solution for solution in current_solutions if dep not in solution.visited_names]

        # check if dep provided by one of the packages already in a solution
        new_not_finished_solutions = []
        for solution in not_finished_solutions:
            if System(list(solution.installed_solution_packages)).provided_by(dep):
                finished_solutions.append(solution)
            else:
                new_not_finished_solutions.append(solution)
        not_finished_solutions = new_not_finished_solutions

        # track deps which may not be deleted
        for solution in current_solutions:
            if dep not in solution.not_to_delete_deps:
                solution.not_to_delete_deps.add(dep)
                own_not_to_delete_deps.add(dep)

        # calc and append new solutions
        current_solutions = finished_solutions
        # used for tracking problems
        new_problems_master: List[Set['DepAlgoFoundProblems']] = []
        found_problems_copy: Set['DepAlgoFoundProblems'] = set(found_problems)
        for solution in not_finished_solutions:

            # add dep to visited names
            # and create another container
            # for problem tracking
            solution.visited_names.add(dep)
            new_problems: List[Set['DepAlgoFoundProblems']] = []

            for dep_provider in dep_providers:
                # way to the package being called in the current solution
                if dep_provider.name not in solution.dict_to_way:
                    way_added = True
                    solution.dict_to_way[dep_provider.name] = own_way[:]
                    solution.dict_to_way[dep_provider.name].append(self)
                else:
                    way_added = False
                # tracking for which deps the package being called has been chosen as provider
                if dep_provider.name not in solution.dict_to_deps:
                    solution.dict_to_deps[dep_provider.name] = set()
                solution.dict_to_deps[dep_provider.name].add(dep)

                # call this function recursively on the dep provider
                # and yield an empty found_problems set instance
                found_problems.clear()
                current_solutions.extend(
                    dep_provider.solutions_for_dep_problem(solution, found_problems, installed_system,
                                                           upstream_system, deps_to_deep_check))
                # save the new problems
                new_problems.append(set(found_problems))
                # remove added things
                solution.dict_to_deps[dep_provider.name].remove(dep)
                if way_added:
                    del solution.dict_to_way[dep_provider.name]

            # reset the problems to the problems
            # we had before calling the dep
            found_problems.clear()
            for problem in found_problems_copy:
                found_problems.add(problem)

            # if there is at least one valid solution
            # problems are not relevant
            # hence add an empty set containing no problems
            for problems in new_problems:
                if not problems:
                    new_problems_master.append(set())
                    break

            # if there are problems contained in all return values
            # show them to the user
            # will most likely be unfulfillable deps in general
            else:
                prob_in_all_ret = set.intersection(*new_problems)
                if prob_in_all_ret:
                    new_problems_master.append(prob_in_all_ret)
                # otherwise append all found problems
                else:
                    new_problems_master.append(set.union(*new_problems))

        # again - at least one valid solution
        # means new problems are not relevant
        if not_finished_solutions:
            for problems in new_problems_master:
                if not problems:
                    break
            else:
                for problem in set.union(*new_problems_master):
                    found_problems.add(problem)

        # filter solutions so that irrelevant solutions are not being
        # used anymore
        # great impact on the performance
        current_solutions = filter_solutions(current_solutions)

    # conflict checking
    for solution in current_solutions:
        # as with dep cycles,
        # conflicts are only relevant
        # if the solution is not already invalid
        if not solution.is_valid:
            continue

        # generate hypothetic system containing the packages of the current solution
        # and check for conflicts with that system
        installed_packages = list(solution.installed_solution_packages)
        conf_system = System(installed_packages).conflicting_with(self)

        # if there are no conflicts, nothing will get deleted, so we may
        # safely assume that we do not get an invalid solution
        if not conf_system:
            continue

        # append the whole current solution to the currently
        # installed system
        # may be empty in case of deep_search
        packages_to_append = solution.packages_in_solution[:]
        packages_to_append.append(self)
        new_system = installed_system.hypothetical_append_packages_to_system(packages_to_append)

        # prepare message for conflict
        additional_message = ""

        # if self cannot be added, this solution
        # is clearly not valid
        if self.name not in new_system.all_packages_dict:
            additional_message = "Tried to install {}, " \
                                 "but it was not possible." \
                                 "".format(Colors.BOLD(Colors.LIGHT_MAGENTA(self.name)))
            is_possible = False
        else:
            is_possible = True

        # these deps have to remain provided,
        # since they are needed for a package which
        # has not been installed yet
        # e.g. A needs B and C, B has been solved with this algo
        # but C not, hence B must remain provided
        # otherwise A cannot be installed
        for dep in solution.not_to_delete_deps:
            if not is_possible:
                break
            if not new_system.provided_by(dep):
                additional_message = "While trying to install {}, " \
                                     "the needed dependency {} has been removed." \
                                     "".format(Colors.BOLD(Colors.LIGHT_MAGENTA(self.name))
                                               , Colors.BOLD(Colors.LIGHT_MAGENTA(dep)))
                is_possible = False
                break

        # same for packages which have to remain installed
        for package in installed_packages:
            if not is_possible:
                break
            if solution.dict_call_as_needed.get(package.name, False) \
                    and package.name not in new_system.all_packages_dict:
                additional_message = "The package {} had to remain installed, " \
                                     "but has been removed.\n" \
                                     "The package which lead to the removal is {}" \
                                     "".format(Colors.BOLD(Colors.LIGHT_MAGENTA(package.name))
                                               , Colors.BOLD(Colors.LIGHT_MAGENTA(self.name)))

                break

        # solution possible at this point if there are no installed packages
        else:
            # check which packages have been removed
            # due to adding the packages
            for package in installed_packages:
                # remove all remainings of the package
                # besides the knowledge that the package
                # has already been built
                if package.name not in new_system.all_packages_dict:
                    solution.installed_solution_packages.remove(package)
                    if package.name in solution.dict_to_deps:
                        for dep in solution.dict_to_deps[package.name]:
                            solution.visited_names.remove(dep)
                        del solution.dict_to_deps[package.name]
                    if package.name in solution.dict_to_way:
                        del solution.dict_to_way[package.name]

            # for the case that there are no installed packages
            if is_possible:
                continue

        # solution not possible!
        solution.is_valid = False
        conflicting_packages = set(conf_system)
        conflicting_packages.add(self)
        ways_to_conflict = []
        for package in conflicting_packages:
            way_to_conflict = solution.dict_to_way.get(package.name, [])[:]
            way_to_conflict.append(package)
            ways_to_conflict.append(way_to_conflict)

        # create the problem
        conflict_problem = DepAlgoConflict(conflicting_packages, ways_to_conflict)
        conflict_problem.additional_message = additional_message
        for way_to_conflict in ways_to_conflict:
            for package in way_to_conflict:
                conflict_problem.relevant_packages.add(package)
        found_problems.add(conflict_problem)

    # we have valid solutions left, so the problems are not relevant
    if [solution for solution in current_solutions if solution.is_valid]:
        found_problems.clear()

    # add self to packages in solution, those are always topologically sorted
    for solution in current_solutions:
        solution.not_to_delete_deps -= own_not_to_delete_deps
        solution.installed_solution_packages.add(self)
        solution.packages_in_solution.append(self)
        solution.visited_packages.remove(self)

    # may contain invalid solutions !!!
    # but also filtered
    return filter_solutions(current_solutions)

Thank you very much for your elaborate response! I'll make sure to take a look at it in greater detail.

A very detailed explanation indeed. You should also put that comment in the wiki of this project. I would be interesting to add some benchmarks to the article too.
Nice work mate!

If versioned dependency support is implemented, it should be done so in aur-rpc. That way higher level tools like aur-deps need minimal changes, and any client-side hacks can be easily removed should the AUR ever support this functionality.

After some consideration, this is the approach I have in mind:

Use a weighted graph in aur-graph, where each node (pkgbase) is provide with a weight (pkgver)
Establish a vertex if the weights between dependency and pkgbase are compatible (i.e. respect >, >=, <=, <, =)

Versions specified in provides should be accounted for. (*)

If a "blank" provides is given, assume pkgver of the provider.
If a versioned provides is given, prioritize it over pkgver of the provider.

The rest of the graph is still constructed, but invalid nodes are printed and exit code is set >0 (compare tsort behavior on dependency cycles)

This should be relatively simple to implement, apart from (*). The benefit is that output from the AUR can be used verbatim instead of strange manipulations in aur-rpc, while all surrounding complexity remains in a single script (here aur-graph). This can be implemented in aur-sync, likely through an optional flag --strict:

Run aur-graph on the cloned build files

If there are errors in constructing the graph, exit with an error.

Verify dependencies that are not part of the graph:

aur-repo-filter: availaibility in the official repos
aur-repo-filter -d custom: availability in a local repository
Ensure versioned dependencies are respected in this step

This should handle the cases by @rmarquis (https://github.com/AladW/aurutils/issues/10#issuecomment-198764936 and https://bbs.archlinux.org/viewtopic.php?pid=1688551#p1688551) and "purely virtual" packages like bemenu at a negligible performance cost:

Using the trick of concatenating .SRCINFO files, aurqueue (aurutils 1.5) went over all 70000 packages in the AUR 3 mirror under 2 seconds.
aur-repo-filter is not as optimized, but should add a delay of ~0.5 seconds in typical use.

The following cases are not handled:

Cyclic dependencies in any shape or form, such as the mingw64 series of packages.

I suppose makedependencies and checkdepends could be classified as "optional edges" to break cycles.

Versioned dependencies specified on the command line. This is purely a pacman feature, and since aur-sync is not a pacman wrapper there's no need to support this.

What I propose is nowhere near as complete as @polygamma algorithm for aurman's, but since with the local repo approach most dependency solving is offloaded to pacman it should suffice in most cases.
Thoughts?

If a "blank" provides is given, assume pkgver of the provider.

libalpm does not do this. Versioned dependencies can not be applied to unversioned depends.

Emails cant update after you edit the post :/. Looks like you may have realised and edited this point away. Just wanted to mention it,

Yeah, I racked my brains over implementing this to finally realize it made no sense at all (e.g. what on provides = foo<=2.0 and depends>1.0). I realized in hindsight that libalpm only allows specific provider versions (=) or none at all.

One case I haven't thought of: https://github.com/actionless/pikaur/commit/14afcd57f79e4886903e350951c8809f11a85f88

Another case which can be quite annoying. -git packages that provide foo=$pkgver.

Yeah. I guess you could download the sources / run makepkg in advance before checking versions...

@Morganamilo That case is broken in pacman anyway when using tagless rN.gH notation.

Seems to work now for intervals (ver1, ver2). For the git stuff I guess I'll wait until someone complains that the error message ("invalid node") is not informative enough...


data distribution

I looked at the author's Github code and found that your test dataset seems to be a dataset obtained from DEM global segmentation. But in this way, your training dataset includes the test dataset. Is it because I misunderstood the code.

The training and test(validation) dataset are generated from random samples of the global map and used separately in the model training. The global map is also processed using the final trained model, since we want to map all craters on Mars.


More accurate mania pp calculation

Not correct according to https://github.com/ppy/osu-performance/blob/master/Src/Processor/Mania/ManiaScore.cpp#L75
The most likely candidate for inaccuracy is the "acc" parameter being only 2 decimals


[Feature Request] Numeric Registers

In a RISCV design register addresses are 5 bit signals which are (most of the time) assigned to their own separate variable. Surfer does not have a 5 bit register address to register name translator (00001 -> "ra"). Hence, I have to manually look up which decoded instruction does a 5 bit number I have encountered 10 stage down the pipeline corresponds to.
Numeric register support would help quite a lot since objdump also supports numeric register names.

Straightforward mapping:
    "zero" -> "x0"
    "ra" -> "x1"
    "sp" -> "x2"
    "gp" -> "x3"

that's super easy, all you have to do is change the register list in the toml file to be "x0", "x1"... instead of "zero", "ra"... and it'll do exactly that

Yea, but toml file is embedded inside Surfer and I can not change it after I build it. I was looking for a simple toggle or an entry in right click menu where you can choose to render it either using alias or numeric names. If you are debugging a new custom RISCV core, it is better to use numeric, but if you are debugging an existing design through virtual jtag and gdb it is better to have it in alias mode. I do believe this feature will be really useful for everyone debugging a new/custom core.
I don't mind patching my tools but we use surfer and gtkwave in a team and we are planning to ditch gtkwave and use surfer exclusively. I hope to keep everything consistent for all since most will use precompiled binaries. However, If this is out of scope of the project or require an overhaul of the codebase I can close the issue.

Ah fair, that makes sense
That's probably more of a surfer issue then, but surfer does now actually allow you to load your own toml files from a config dir, see https://gitlab.com/surfer-project/surfer/-/merge_requests/385
so you could do it that way for now

My feeling with Surfer's default RISC-V decoders is, that those should be just a convenient way to use quickly.
If you are working on your own core you probably want to create a config directory with just the supported extensions + your custom instructions. There you can rename the registers. I know renaming them is cumbersome, maybe we can just provide two versions of toml files in this repo.

Yes, custom configs is the right way to approach this issue. Especially if I can create a .surfer directory in the repo/project root as requested in  https://gitlab.com/surfer-project/surfer/-/issues/271. I will go ahead and close this issue for 271.


Smapp force terminated.

Environment

Smapp version: 1.1.2
OS: Ubuntu 22

Describe the bug
After a few hours of starting Smapp, it is forced to terminate.
Steps to reproduce

Start Smapp
A few hours elapsed
Smapp force terminated

Expected behavior
Smapp stays up and running
Actual behavior
Force terminated in a few hours.
Because of this, this node has been unable to earn mining rewards for a long time.
Logs
log.txt.zip

Can someone check and fix?

@hryksbt Sadly there is nothing interesting in these logs.
Could you check the latest Smapp v1.1.9 and tell me if it is still actual?
Then please upload app-logs, and it will be helpful if you will take a look at process CPU and RAM consumption (and mb take a profiler snapshot). Otherwise, please close the issue.

@brusherru It is still occurred in 1.1.9.
CPU load is mostly 100% and RAM is about 3GB used that total is 16GB.
Log data is same.

applog-1.1.9.txt
Here is 1.1.9 applog.
Thanks.

@hryksbt Sadly there are no clues in the logs, and we can't reproduce it.
Could you provide more info:

Check out /var/logs/syslog and /var/logs/apport.log. If you find there is something about Spacemesh app — please attach such a part. Since the system terminates Smapp for some reason — one of these log files should contain an actual error (most likely apport.log).
Your system specs (CPU, GPU)
Your PoS data is created, HDD/SSD is always accessible (and it is not an external drive that goes sleep sometimes), and you're smeshing. Right?

I found below error in /var/log/syslog.

/var/log/syslog:Sep 20 18:07:39 i7k03-4090 systemd-oomd[826]: Killed<EMAIL_ADDRESS>due to memory pressure for<EMAIL_ADDRESS>being 50.86% > 50.00% for > 20s with reclaim activity

Intel i9-13900K, i5-13600K, RTX4070ti, RTX4080, RTX 4090, RTX3070, RTX3080

Yes. Because some PS's disks are including OS.
I have some case like external USB or SATA internal too.
All the same situation.

This is especially occurred only Ubuntu. When epoch round finish, mostly all PCs are down in spite of any OS.
Then start Smapp, Ubuntu will mostly continue to down.
And this is memory swap condition.
$ swapon -s Filename				Type		Size		Used		Priority /swapfile                               file		2097148		229628		-2
This has lots of free space.

$ free
total        used        free      shared  buff/cache   available
Mem:        16217676     5460576      169524       20084    10587576    10392104
Swap:        2097148      752852     1344296

I'm trying to remove swap disk now.
It seems better than before.
I am going to check and then update result.
If this is solution, I recommend to you write remove swap information especially for Ubuntu.

@hryksbt did you find a solution?
It seems like a problem with your environment, not Smapp or go-spacemesh.
Probably something is broken in /etc/systemd/oomd.conf or maybe you have some weird policy that allows any app to take up to 50% of memory, which may cause the problem. In the rest, I have no idea what else to check :(

It works much better after killed swap.
But still I get force termination.
Anyway, I will close this.
If I find cause of issue, I might re-open ticket.


Quick access

Is your feature request related to a problem? Please describe.
I like the possibility to have access to frequently used folders without having to look for them manually.
Describe the solution you'd like
Include a Quick access area in the sidebar, allowing to pin folders and to automatically display the most-used ones
Describe alternatives you've considered
None
Additional context

@MarcAnt01 You can already pin a folder the sidebar by right clicking on a folder and selecting 'Pin to sidebar'.

@MarcAnt01 You can already see recently opened folders on the jumplist, however, adding a quick access section is something we can look at doing as well.

Merging with #692.


Add mem unexpected behaviour

The add_mem phase behaves differently, depending on surrounding externals.
Even if these externals are never called and to not interfere with the functions.
Example: https://github.com/NeuralCoder3/thorin2/blob/ho_codegen/lit/mem/add_top_mem.thorin
Example2: https://github.com/NeuralCoder3/thorin2/blob/ho_codegen/lit/mem/add_top_mem2.thorin
In add_top_mem, the call remains f (⊤:%mem.M, argc_1111242, return_1111250).
In add_top_mem, the call is changed to f (_1113823, argc_1113827, return_1113835).
The only difference between the two examples are the externals g and h (h seems to be the important one).
It seems like mems in application argument position are sometimes taken in consideration as current memory.
However, we should only consider vars and application results.

One problem was that the else case where mem as result from any operation is registered in val2mem.
I think we do not need this case.
But this does not explain why other externals influence the result.

The behavior was probably caused due to the sharing of the two ⊤:%mem.M instances.
The fix is in the ho_codegen branch (the only place where it is currently needed) and will be merged with it.


Example of progressive timeouts (i.e. FastFail)

When making requests to hosts the first request attempt should limit the timeout to that 99 percentile (or other appropriate threshold).  Using ocelli this responsibility will likely be place on the submit/retry plan.  At minimum we should provide an example of how this can be implemented.

Progressive timeouts is a poor name for this feature.  We'll focus on #66.


Minor changes

Codecov Report

Merging #87 (82bad73) into master (eab10b0) will decrease coverage by 0.20%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master      #87      +/-   ##
==========================================
- Coverage   78.11%   77.91%   -0.21%
==========================================
  Files          42       42
  Lines        1549     1553       +4
==========================================
  Hits         1210     1210
- Misses        339      343       +4

Impacted Files
Coverage Δ

...ELTation/DIFramework/Runtime/ContainerLifecycle.cs
93.90% <0.00%> (-4.82%)
:arrow_down:


Solasta has detected an instability

This error happens every time I start the game.
Attached the log file.
The log file also mentions some specific character files, which I also attached.
Player.log
Korragos.zip
Henry.zip
Solasta 1.5.97
UB <IP_ADDRESS>

At some point you had a wrong umm version installed which broke one or hero files. Henry is one of them. Need to get rid of this file but while game is open.


upstream: per host connection limit support?

Title: per host connection limit support?
Description:
using this feature to prevent too much active connections in a host.
diff --git a/api/envoy/config/cluster/v3/cluster.proto b/api/envoy/config/cluster/v3/cluster.proto
index f4e3386cf..08accac6e 100644
--- a/api/envoy/config/cluster/v3/cluster.proto
+++ b/api/envoy/config/cluster/v3/cluster.proto
@@ -756,6 +756,9 @@ message Cluster {
   // parameter to 1 will effectively disable keep alive.
   google.protobuf.UInt32Value max_requests_per_connection = 9;

+  // Optional maximum connections for upstream host.
+  google.protobuf.UInt64Value max_connections_per_host = 70;
+
   // Optional :ref:`circuit breaking <arch_overview_circuit_break>` for the cluster.
   CircuitBreakers circuit_breakers = 10;

diff --git a/source/common/conn_pool/conn_pool_base.cc b/source/common/conn_pool/conn_pool_base.cc
index a5df5aadc..9b7a76600 100644
--- a/source/common/conn_pool/conn_pool_base.cc
+++ b/source/common/conn_pool/conn_pool_base.cc
@@ -44,6 +44,11 @@ bool ConnPoolImplBase::shouldCreateNewConnection(float global_preconnect_ratio)
     return pending_streams_.size() > connecting_stream_capacity_;
   }

+  if (host_->cluster().maxConnectionsPerHost() > 0 &&
+      host_->stats().cx_active_.value() > host_->cluster().maxConnectionsPerHost()) {
+    return false;
+  }
+
   // If global preconnecting is on, and this connection is within the global
   // preconnect limit, preconnect.
   // We may eventually want to track preconnect_attempts to allow more preconnecting for

Hmm, your code change snippet does not match the issue description. From the description it looks like you want per host circuit breaker but your code changes preconnect behavior. Can you describe a bit more what you are trying to accomplish, please?

Hmm, your code change snippet does not match the issue description. From the description it looks like you want per host circuit breaker but your code changes preconnect behavior. Can you describe a bit more what you are trying to accomplish, please?

@yanavlasov thanks for you reply.  Yes, per host circuit breaker is more flexible and complex, it will support our requirement.  In my case, we want limit per host connections by cluster level config, all host has the same limit.

If all hosts have the same limit, why not then set the cluster circuit breaker limit to number_of_hosts * max_connections_per_host ? This may work if the number_of_hosts stays the same. If loadbalancing works correctly you should have the same number of connection per host.

If all hosts have the same limit, why not then set the cluster circuit breaker limit to number_of_hosts * max_connections_per_host ? This may work if the number_of_hosts stays the same. If loadbalancing works correctly you should have the same number of connection per host.

It could easily burst connections to number_of_hosts * max_connections_per_host when enable session sticky.

Ok, adding @alyssawilk for any more comments in this area

Happy to help.  @arikachen could you give a concise explanation of what you're hoping for here?

@alyssawilk
Now the circuit breaker supports the limit of the maximum number of connections at cluster granularity.
But for each host, there is no separate limit on the number of connections. Although a reasonable load balancing algorithm can be configured to ensure that the maximum number of connections per host is roughly the same (The maximum number of connections in the cluster circuit breaker / The number of hosts), this restriction is not strict.
We hope that for each host, a strict maximum connection limit can be configured just like a per host circuit breaker.

Gotcha.  Yeah that seems a reasonable request and I don't think there's any current bounds today, and especially not cross-thread.
cc @RyanTheOptimist who was interest in global per-host limits too.


Investigate using GetAccountAuthorizationDetails to augment/replace IAM watchers.

http://boto3.readthedocs.io/en/latest/reference/services/iam.html#IAM.Client.get_account_authorization_details
'User'|'Role'|'Group'|'LocalManagedPolicy'|'AWSManagedPolicy',
This single call may be able to replace many of the calls currently made in the watchers for user, role, group, and managed policies.

Cool idea, but with the bananapeel architecture, this sort of optimization is not necessary.


new PR of "Description" functionality

This is new pr for issue #255 ,
This pull request is sync with your base repo.
All merge conflicts are handled.
No significant changes in from origin repo was encountered by merging new commits, everything is fine...

is responsiveness handled?

is responsiveness handled?

It wasn't enough responsive from beginning... But I tested if it was, and it was quite responsive... Not full but quite responsive... Because of css from external framework I couldn't work on it, the original css was kept messing... So I didn't touched it

is responsiveness handled?

no please close this pull request, i have accidently ignored a big issue, need to fix it now. Please dont merge.


Add support for additional-apks option

Fixes #
Checklist

[x] Documented
[x] Unit tested
[x] release_notes.md updated

Codecov Report

Merging #695 into master will increase coverage by 0.27%.
The diff coverage is 94.04%.

@@             Coverage Diff              @@
##             master     #695      +/-   ##
============================================
+ Coverage     77.48%   77.76%   +0.27%
- Complexity      642      645       +3
============================================
  Files           112      112
  Lines          2594     2635      +41
  Branches        368      369       +1
============================================
+ Hits           2010     2049      +39
  Misses          347      347
- Partials        237      239       +2

Remember to set the ZenHub milestone/estimate/epic if there's not a linked ticket that already has this info. I updated this PR.


docs: ics27 v6 documentation updates

Description

Adding deprecation notice to Authentication Modules page and moving to bottom of list
Removing occurrences of Authentication Modules in various doc pages
Moving Active Channels page further down the list
Adding Messages doc

Documents MsgRegisterInterchainAccount and MsgSendTx

part of: #1948

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

[ ] Targeted PR against correct branch (see CONTRIBUTING.md)
[ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
[ ] Code follows the module structure standards.
[ ] Wrote unit and integration tests
[ ] Updated relevant documentation (docs/) or specification (x/<module>/spec/)
[ ] Added relevant godoc comments.
[ ] Added a relevant changelog entry to the Unreleased section in CHANGELOG.md
[ ] Re-reviewed Files changed in the Github PR explorer
[ ] Review Codecov Report in the comment section below once CI passes

Also is it intentional that the package-lock.json is removed?

I didn't remove it, but it does look like running make view-docs locally had it building and seemed to remove a large chunk of lines. Not entirely sure why.


Cluster interface and DataSourceCluster bindings

The Cluster interface provides a consistent abstraction for interacting with various database-related resources based on their role in the underlying database topology.
Apply that abstraction to the DataSourceModule, binding DataSources indirectly via DataSourceClusters.

IMO T - can default to the writer if no read specific endpoint available
On Wed, May 16, 2018 at 4:48 PM Joey Muia<EMAIL_ADDRESS>wrote:

@jmuia commented on this pull request.
In misk/src/main/kotlin/misk/jdbc/Cluster.kt
https://github.com/square/misk/pull/194#discussion_r188767620:

+package misk.jdbc

+/**

[Cluster] provides a consistent abstraction for

interacting with various database-related resources [T]

based on their role in the underlying database topology.

Database clusters are often composed of a Read/Write master

instance and a set of Read-Only replica instances.

Examples include a Read-Only [DataSource] or a Read/Write

[SessionFactory] in Hibernate.

*/
+interface Cluster {
fun writer() : T
fun reader() : T

should this be T??
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/square/misk/pull/194#pullrequestreview-120813846, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAX-WVNrj6q5Egt_I9LTk54j0UkU-m2sks5tzJCEgaJpZM4UCApD
.

Do we potentially want different types for reader vs writer? (I’m thinking
ReadingTransacter vs Transacter)
On Fri, May 18, 2018 at 9:54 AM Joey Muia<EMAIL_ADDRESS>wrote:

@jmuia commented on this pull request.
In misk/src/main/kotlin/misk/jdbc/Cluster.kt
https://github.com/square/misk/pull/194#discussion_r189275909:

+package misk.jdbc

+/**

[Cluster] provides a consistent abstraction for

interacting with various database-related resources [T]

based on their role in the underlying database topology.

Database clusters are often composed of a Read/Write master

instance and a set of Read-Only replica instances.

Examples include a Read-Only [DataSource] or a Read/Write

[SessionFactory] in Hibernate.

*/
+interface Cluster {
fun writer() : T
fun reader() : T

One thing to consider: if you don't provide a specific reader then you'll
likely use @Inject @Exemplar DataSource directly rather than @Inject
@Exemplar DataSourceCluster anyways. This saves having to null-check the
reader.
—
You are receiving this because your review was requested.
Reply to this email directly, view it on GitHub
https://github.com/square/misk/pull/194#discussion_r189275909, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAX-WRc_43B9e7rHDfOj3q-tCVLZc2H8ks5tztKVgaJpZM4UCApD
.

A Cluster as defined here is specifically not a Vitess cluster - it is an
unsharded collection of database instances (master, sync/semi-sync standby,
zero or more local read replicas) pointing to the same actual dataset. It
is equivalent to an RDS cluster or ODS cluster.
On Mon, May 21, 2018 at 9:32 AM Jon Tirsen<EMAIL_ADDRESS>wrote:

@tirsen commented on this pull request.
In misk/src/main/kotlin/misk/jdbc/Cluster.kt
https://github.com/square/misk/pull/194#discussion_r189587476:

interacting with various database-related resources [T]

based on their role in the underlying database topology.

Database clusters are often composed of a Read/Write master

instance and a set of Read-Only replica instances.

Misk defers management of these replicas to the underlying

persistence layer and requires a single addressable endpoint.

Examples include a Read-Only [DataSource] or a Read/Write

[SessionFactory] in Hibernate.

N.B. If a Read-Only resource is not provided Misk will

default to the Read/Write resource.

*/
+interface Cluster {

So a cluster is like a vitess cluster?
In misk/src/main/kotlin/misk/jdbc/Cluster.kt
https://github.com/square/misk/pull/194#discussion_r189588212:

Database clusters are often composed of a Read/Write master

instance and a set of Read-Only replica instances.

Misk defers management of these replicas to the underlying

persistence layer and requires a single addressable endpoint.

Examples include a Read-Only [DataSource] or a Read/Write

[SessionFactory] in Hibernate.

N.B. If a Read-Only resource is not provided Misk will

default to the Read/Write resource.

*/
+interface Cluster {
val writer: T
val reader: T

With Vitess we would not need to have separate readers and writers. You
can have a single connection and change whether you're reading from
replicas or from the master with a USE statement. USE @replica makes your
connection start doing replica reads, USE @master takes you back to the
master.
I'm imagining we would instead have methods on the transacter that looks
like this:
transacter.replicaRead().executeWithBackoff(s -> {
s.createQuery(...)
...
})
(Sorry not valid Kotlin, I'm still a noob there.)
And this for master reads:
transacter.consistentRead().execute(...)
For slow OLAP style queries:
transacter.replicaRead().slow().execute(...)
And even in the future:
transacter.consistentReplicaRead().execute(...)
If Vitess implements this:
http://www.proxysql.com/blog/proxysql-gtid-causal-reads
/cc @mmihic https://github.com/mmihic
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/square/misk/pull/194#pullrequestreview-121794659, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAX-WalOV0un5HT4NRQK4K0UT01rNZVdks5t0sHogaJpZM4UCApD
.


Issue with in k8s.io/docs/concepts/cluster-administration/cloud-providers/ -- openstack section

This is a...

[ ] Feature Request
[x] Bug Report

Problem:
In configuring k8s v1.11 for the OpenStack with kubenet network plugin, when using [Router] section of the cloud provider config file, then an error occurs in kubelet such as:
can't store data at section "Router".
Proposed Solution:
This problem can be resolved by using follow:
[Route]
router-id=<router-id>

Page to Update:
https://kubernetes.io/docs/concepts/cluster-administration/cloud-providers/

11389