PUT a Resource with different links

Hi,
I would ask some clarification about library usage best practice in resource and _links update.
I have a resource coming from server like this:
{
   "attribute": "oldValue",
   "_links": {
          "self": "...",
          "related": "{"href": "/resoucesClassB/2" }",
          "another": "{"href": "/pinuts/3"}"
    }
}

I think it's correct have a resource "related" directly in the link, is it?  ("/resourceClassB/2" it's the url to a specific resource)
I would like to update this resource on the server, and change both related resource (change a relationship) and the attribute.
Suppose the server does not support PATCH, so I need to use PUT.
In a case like this, when I PUT a resource, should I add all its links? (or it's a wrong practice in your point of view?)
Because Hal resource as implemented are immutable, I can't change their values, so I need to do a copy.   Because I need to change also the links, i need to add them to the copy.
(maybe I'm wrong, please help me in case i'm going in the wrong way)
var resource = ...;
var relatedResourceB3 = ....;
var selfB3= relatedResourceB3.$href('self');    //  e.g. "/resourceClassB/3"

var resourceCopy = angular.copy(resource);
resourceCopy.attribute = "new value";
resourceCopy._links = { "related": {"href" : selfB3}};

resource.$request().$put('self', null, resourceCopy);

How can I clone a resource, with all its link, to be able to change them? (in this case "another" link is missing), and which is the correct way to change them?
Can I build a "new resource" (not immutable) starting from an existing resource, as a mutable clone?
var mutableClone = resource.$mutableClone();
mutableClone.attribute = "new value";
mutableClone.$put();   //with no parameter it could put on self.

Hi @francescobragagna,
This is a good question, which I can't really answer.
If you use a PUT request, you should always send all the whole resource.
How the whole request should look like I can't answer. I had the same problem with JSON Patches and still don't know how to solve them.
I think this is something that should be clarified by https://github.com/mikekelly/hal_specification

Also if I say If you use a PUT request, you should always send all the whole resource., that doesn't mean that the links have to belong to the resource. But then the question is how to Link them? Link Headers?

"I think this is something that should be clarified by https://github.com/mikekelly/hal_specification"
I've already asked similar question here:
https://github.com/mikekelly/hal_specification/issues/5
and
https://groups.google.com/forum/#!topic/hal-discuss/r6T_gTCSBDk
I was expecting more 'common behaviours' on resource update. Because when I need to browse entities there are no problem.
The problem starts when you need to unlink, link multiple, link existing beween them, and so on, which are in my point of view  common operation on an application with many resources with relationships N:M, 1:N, ... to be maintained.
And in my point of view this problem is related to the meaning of "self" link in an embedded resource.

In the google groups I think I've read most of the post (about 400) and somethimes the discussions are going into investigation of self meaning, and resource update. But Mike many times answers it's not a matter of HAL specification define how to update resources.
I think is partially correct, but partially not.
I've investigated also usage of some of the [public APIS ](https://github.com/mikekelly/hal_specification/wiki/APIs)listed in the Mike's github: most of them doesn't update resource, or simply delete/build again them. It's still not clear to me

Since the HAL specification does not specify anything for this matter, I don't think that we'll implement any opinionated solution in this library. Please create a new issue for either an implementation as soon as the specification has a solution or a way for the user to extend the library with opinionated solutions.


ingest Wagner members and maybe more from Myrotvorets

https://myrotvorets.center/criminal/ has a bunch of detailed records of Russian soldiers, PMC Wagner members and other criminals.
There are detailed records with photos, army unit, passport data, etc: eg see https://myrotvorets.center/criminal/feofilaktov-aleksej-pavlovich/ .
I don't know whether this list is available as CSV, but 4k records of PMC Wagner members are available:

https://myrotvorets.news/spysok-naemnykov-y-terrorystov-chvk-va/ (page)
https://myrotvorets.news/wp-content/uploads/2021/12/vagner4000.zip (CSV)
https://myrotvorets.news/wp-content/uploads/2021/12/vagner4000.xlsx (Excel)

The excel has 3 columns of names per person. For some persons, there are additional rows that give additional names. You can identify them by the URL of the detailed information. Eg all these rows mention the same detailed page as cited above:

Feofilaktov Aleksej Pavlovich	Феофилактов Алексей Павлович	Феофілактов Олексій Павлович
Feofilaktov Aleksej Pavlovich	Феофилактов Алексей Николаевич	Феофілактов Олексій Павлович
Feofilaktov Aleksej Pavlovich	Феофилактов Алексей Павлович	Феофілактов Олексій Миколайович
Feofilaktov Aleksej Nikolaevich	Феофилактов Алексей Павлович	Феофілактов Олексій Павлович

Number of total and deduplicated rows (13.4% duplication):
# wc -l vagner4000.csv
4496 vagner4000.csv

# cut -d, -f6 vagner4000.csv | sort | uniq | wc -l
3965

Commented at https://myrotvorets.news/spysok-naemnykov-y-terrorystov-chvk-va/ :

I posted a task https://github.com/opensanctions/opensanctions/issues/325 to ingest this list to https://www.opensanctions.org/. Two questions:

What is the total number at https://myrotvorets.center/criminal/?
Are there more CSVs that list people from these pages?

Here we are: https://www.opensanctions.org/datasets/ru_myrotvorets_wagner/
It would of course be lovely to get more detail in the CSV (e.g. birth place, specific accused crimes)

Thanks @pudo !
Could you add links "member" to https://www.opensanctions.org/entities/Q36597284/ ?
Can you fix these IDs (they are broken links due to some tech problems in WD):

Q120706615 -> Q120851450 (ZHemchuzhnyj Andrej Valterovich)
Q120712921 -> Q120854799 (Ermolaev Aleksandr Elizarovich)

On the other hand, we hope to add OS ID to WD soon.

get more detail in the CSV (e.g. birth place, specific accused crimes)

That'd be hard since the myrotvorets descriptions are free-text and show great variety.
(There's no birth place, but there is Address)


anyone achieve successfully run enroll.py and other python files please help i am wokring with python in Windows

Please provide more info on the problem

When running viewer.py it is showing camera feed but after pressing key "a", "e", "d" it is terminating
Version: 0.23.0
[2021-08-02 15:20:13.369] [debug] [Utilities] detected capture device.
[2021-08-02 15:20:13.471] [debug] [Preview] Preview started!
[2021-08-02 15:28:18.172] [debug] [WindowsSerial] Opening serial port COM9
[2021-08-02 15:28:18.173] [error] [FaceAuthenticatorImpl] Failed to open serial port. GetLastError: 2
[2021-08-02 15:28:18.174] [debug] [NonSecureSession] Close session
Traceback (most recent call last):
File "viewer.py", line 238, in
main()
File "viewer.py", line 234, in main
input_loop()
File "viewer.py", line 127, in input_loop
cmd_exec.get(chr(keycode), lambda: None)()
File "viewer.py", line 83, in auth_example
with rsid_py.FaceAuthenticator(PORT) as f:
RuntimeError: Error

I think you should check serial port of camera. In samples code, default PORT='COM9'. If u using Windows, you can identify serial port of connected device in Device Manager.

[2021-08-16 10:39:08.992] [debug] [WindowsSerial] Opening serial port COM3
User id to enroll: 12
[2021-08-16 10:39:12.742] [debug] [NonSecureSession] Start session
[2021-08-16 10:39:12.743] [debug] [PacketSender] Sending packet 'o'
[2021-08-16 10:39:12.747] [debug] [PacketSender] Waiting packet..
[2021-08-16 10:39:12.753] [error] [PacketSender] Got invalid crc. Expected: 561. Actual: 20100
[2021-08-16 10:39:12.754] [error] [NonSecureSession] Failed to recv device start session response
[2021-08-16 10:39:12.754] [error] [FaceAuthenticatorImpl] Session start failed with status 108
on_result EnrollStatus.CrcError
Traceback (most recent call last):
File "enroll.py", line 27, in
f.enroll(user_id=user_id, on_hint=on_hint, on_progress=on_progress, on_faces=on_faces, on_result=on_result)
RuntimeError: CrcError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "enroll.py", line 31, in
print('Users: ', users)
TypeError: exit(): incompatible function arguments. The following argument types are supported:
1. (self: rsid_py.FaceAuthenticator, arg0: capsule, arg1: capsule, arg2: capsule) -> None
Invoked with: <rsid_py.FaceAuthenticator object at 0x000002D0891FB2F0>, <class 'RuntimeError'>, RuntimeError('CrcError'), <traceback object at 0x000002D0892194C0>
[2021-08-16 10:39:12.761] [debug] [NonSecureSession] Close session
[2021-08-16 10:39:12.765] [debug] [WindowsSerial] Closing serial port

any idea for CRC error


renaming the daily_prayer class to prayer_info

@hamir
this PR should replace the previous one that encountered many conflicts and was hard to fix up.

Btw you know you can have a local branch without pushing it to remote right?

yes i know, but why would that be useful here?


feat: add support for fuzzy matching in full-text search

This PR address this Github issue by adding support for fuzzy matching in Full-Text search.
To add fuzzy matching support, the function definition for match and matches methods have been modified
from
match(value: string | number | boolean): Search
matches(value: string | number | boolean): Search

to
match(value: string | number | boolean, options?: { fuzzyMatching?: boolean; levenshteinDistance?: 1 | 2 | 3 }): Search
matches(value: string | number | boolean, options?: { fuzzyMatching?: boolean; levenshteinDistance?: 1 | 2 | 3 }): Search

The fuzzyMatching parameter is used to enable fuzzy matching and the levenshteinDistance parameter is used to define the Levenshtein distance for fuzzy matching.
For more info about fuzzy matching with Redis please checkout the Redis docs

Probably gonna do a publish with a couple of community PRs in the next week or so. This will be amongst them. Thanks!

Merged and published. https://www.npmjs.com/package/redis-om


error in displaying plots in comments pr corrected

Retail TimeSeries Forescast Report
Path                 cv_metric    mape_metric    mse_metric          rmse_metric
report_metrics.json  4.06768      3.04859<PHONE_NUMBER>996.2783  1908643.02844
Historical Forecast

Error Distribution

Testing Forecasting


add github action support

cc @fgaz @domenkozar

What's missing here?

I have a file called a file called non-broken.nix that can be called directly by nix-build-uncached, while this repo has an attribute cached-outputs in ci.nix. I guess one could add a wrapper that just imports this attribute from ci.nix.

I can refactor this now, since nix-build-uncached became smarter.

All good?


refactor: simplify sheet loading related code

A bit of refactoring to have a single entrypoint for sheet loading and make future feature development easier

@PrettyWood curious what you think about this. Commits should be pretty independent so I can revert one or several in case you disagree with some decisions


Check for inconsistencies within the fields contents

For example, if games have the same inspired by … they may also have the same category.
Or if they have an asset license, they may be open content
Of if they are a remake/clone they should also have an inspired by.
Or if they have Godot/LÖVE as dependencies they probably have Windows, Linux, macOS as platforms, if they have LibGDX as dependency, it's also Android.
Languages go as well together preferably with certain dependencies, build systems and platforms (JavaScript <> Web; C,C++ <> Make, CMake, SDL; Java <> Gradle, Python <> pygame).
Get them automatically from the entries and then kind of check and warn if unusual patterns are detected.

I now parse the entries and can read and write them. I rewrite the maintenance tasks and this can be one of them.


is it possible to add custom columns in the task list table?

Yes, by overriding providing the default TaskListTable and TaskListHeader functions. You will need to extend the Task interface if you want to add new columns in the data that you provide. See the styling options in the documentation at the bottom Link

can you give an example code , because I didn't understand what you said above.

can you give an example code , because I didn't understand what you said above.

take a look at these two files as references:
task-list-table.tsx
task-list-header.tsx
You can implement ur own components and render whatever you want. Just provide ur custom components as TaskListHeader and TaskListTable according to GanttProps interface public-types.ts


Custom Attributes from config

This PR enables user defined attributes from a yaml configuration file.  Attributes can be by group or individual file or both.

@jpower432 Can you give this a preview and let me know if the approach makes sense before I remove the WIP?

closes #20


Dropdown don't open

Hi,
I just created "hello world" meteor application, and copy&pasted example menu from http://semantic-ui.com/collections/menu.html

    Home

    Messages

      Language

        English
        Russian
        Spanish

Sign Up

It's correctly styled, but dropdown doesn't open (Linux/Chrome).
Do I need to call any jQuery initialization code on Template.rendered ?
Thanks!
Petar

It behaves the same in FireFox too - I cannot open dropdown.

Yes, exactly – call .dropdown() on the element inside the rendered hook.

This fixed it for me:
Template.layout.created = function() {
  $('body').delegate('.dropdown', 'click', function(event) {
    $('.dropdown').dropdown();
  });
};

layout is my main template. There probably is  better solution than calling it every time it's clicked.

I had to do this:
Template.myTemplate.onRendered(function () {
  $('.ui.dropdown').dropdown();
});

The solution used by @sungwoncho is the correct one! :+1:
Doing the same on parent templates won't guarantee the same results since child templates might still not be ready when the onRendered hook of the parent is triggered...

This should be something to add to the Readme.md file to get people get started the right way...
CC: @flemay

See also #16 and this other comment I posted last January...

Thank you folks! :)
Closing this issue.


feat: add Azure FunctionApp Module

This PR is for adding Azure Function App module to the Terratest.
With this module, it can be possible to test Azure Function App resources on Terraform scripts.

Hi @yorinasub17, PR is updated and ready for your review. Here is the related CI. the failure in it is unrelated to the change, but it is due to Azure failure  to create cosmos DB due to unavailability


Click with promise.All

I'm testing site navigation with Playwright codegen mode (https://playwright.dev/docs/codegen) and I note that some click produce something like:
await Promise.all([
   page.click('text=Confirm'),
   page.waitForNavigation(),
]);

I have a spefic page where this is importat because of some async RestAPI calls.
How could be handle it with HBrowser?

Something like this should do it i think:
playwrightWindow.getJsContext().executeJavaScript(""+
"await Promise.all([\n"+
  " page.click('text=Confirm'),\n"+
   "page.waitForNavigation(),\n"+
]);");


Update NPM with fix for index.js

The latest 1.0.4 from npm is still showing the following code.
require('dist/angular-nvd3.js');
module.exports = 'nvd3';
Can you update the latest copy there so that we can move from 1.0.3 to 1.0.4?

:+1:
I ran into this when I tried to webpack with angular-nvd3.  I see this is fixed in the GitHub repo, but it still needs to be republished to npm.

Done, thanks.
index.js is fixed in v1.0.5 (since npm don't allow to change old releases).


unifying .spec.selector spelling

unifying .spec.selector spelling

/assign

@Bradamant3 i did a ag -s  '`spec\.'  search for the same formatting problem.  It seems that there are still many files need to be fixed. I will fix them later because it needs a carefully review and i don't want to send a big pull request .

@lovejoy thank you so much for your care and attention -- and also for your thoughtfulness about PR size. Whenever you can get to other related changes is great.
/lgtm
/approve


Change workflow files

Remove rust-cache
Replace setup-rust-action by rust-toolchain
Run others checks only if the build step didn't fail
Run checks on other platforms than linux if the linux job didn't fail (PR only)


Amount Preferences Fixes

Fixes: #129
Changes

getPreferences checks input amouts against the amounts of the keys
keepPreferences and sendPreferences as different optional parameters in createSplitPayload.
modify splitAmount behaviour -in case of no specified preference- to create amounts not from powers of 2
but from the amounts of the keyset instead.
splitAmount reorders its outputs by descending/ascending order.

PR Tasks

[x] Open PR
[x] run npm run test --> no failing unit tests
[x] run npm run format

Now the changes are isolated.

Maybe this is out of scope for this PR but we should also get rid of this behavior of the lib where it overwrites amount if preference is set. We should throw an error instead of overwriting things and force the implementor to properly use the library.
Example:
if (options?.preference) {
    amount = options?.preference?.reduce((acc, curr) => acc + curr.amount * curr.count, 0);
}

looks like this PR needs some conflict resolution. What's the status on this? @lollerfirst

looks like this PR needs some conflict resolution. What's the status on this? @lollerfirst

Conflicts resolved and I made this into a non-breaking change. Now issues a deprecation warning and converts Array<AmountPreference> to Preferences when needed.

@gandlafbtc Thank you! I applied the suggestions.


Currency dropdown

Added initial currency dropdown to overview page
https://issues.redhat.com/browse/COST-1808

Codecov Report

Merging #2218 (780cd1d) into master (52b933d) will increase coverage by 0.02%.
The diff coverage is 81.81%.

@@            Coverage Diff             @@
##           master    #2218      +/-   ##
==========================================
+ Coverage   71.58%   71.61%   +0.02%
==========================================
  Files         393      393
  Lines        7197     7194       -3
  Branches     1568     1563       -5
==========================================
  Hits         5152     5152
+ Misses       2014     2011       -3
  Partials       31       31

Impacted Files
Coverage Δ

src/api/currency.ts
28.57% <33.33%> (+3.57%)
:arrow_up:

src/store/currency/currencyActions.ts
100.00% <100.00%> (ø)

src/store/currency/currencyCommon.ts
100.00% <100.00%> (ø)

src/store/currency/currencySelectors.ts
100.00% <100.00%> (+25.00%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 52b933d...780cd1d. Read the comment docs.


Is that possible?
Thanks 👍.

Ok thanks!

@vcalderonoyarce I'm closing the issue considering it worked, feel free to re-open if it does not work


Abandon CircleCI in favor of github actions

Motivation
All the falcosecurity org moved to github actions, retiring CircleCI. We should move away libs from CircleCI too, for consistency.
For now, circleCI is just used to test arm64 support since it provides arm64 machines.
Therefore we first need an arm64 runner; either we wait for github actions to finally provide linux arm runners or we use our own test-infra nodes (like we do for Falco).
This is just to track the issue.

/assign

See https://github.com/actions/runner-images/issues/5631

yeah absolutely we cannot remove circle CI until we have an arm node :/

/hold

Since we started using the CNCF Community bare metal cluster for kernel testing, it could be a good resource that we could consider. While we already have approval for the kernel testing project that we're running, I wanted to make sure we could start another node if necessary and if we need to ask permission beforehand. This is the issue on the topic: https://github.com/cncf/cluster/issues/251

I'll note that it's relatively expensive for the CNCF CIL to have an entire 80-core bare metal node used 24/7 just for occasional sporadic Github Actions runs.

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

/remove-lifecycle stale

I'll note that it's relatively expensive for the CNCF CIL to have an entire 80-core bare metal node used 24/7 just for occasional sporadic Github Actions runs.

I agree! The kernel-testing framework  will, sooner or later, be enabled on PRs, master branch and releases, concurrently.
It's just that the tool we use today to manage firecracker vms (ie: https://github.com/weaveworks/ignite) does not support concurrent runs.

cc @Andreagit97
latest kernel-testing updates brought a new ubuntu test image that uses the exactly same image used by our remaining circleCI job: https://github.com/falcosecurity/kernel-testing/commit/3f359211f1155a3cf91046bef7befc914505c4c6
Even if we only run scap-open and not drivers_test for the moment, i think we can safely kill the last job and drop circleCI (after all we are concerned about verifier issues).
WDYT?

yes makes sense! i think we can drop circleCI


Introduce a resource releaser method for SerializationProvider as a precursor to the real fix for #635

The responsibility of closing a persistent serializer has been moved to the SerializationProvider via the releaseSerializer method. This way we achieve consistency in handling the life cycle of the provided objects.
Serializers are not Closeable anymore. So caches or rather the serialization provider won't be closing the serializers when they are closed.

Check-in incorporating review feedback.

test this please


Optimize integer reading using bmi2 instructions

Unclear how big the benefit actually would be. Many integers and especially enum identifiers would be small so the small bytewise loop should actually be faster. The optimization would require and additional branch to make sure we can read 4 or 8 bytes and fallback if not.
For writing to a vec, bmi2 could be more beneficial since could always write bigger chunks and then adjust the len afterwards.

For reference, code for branchless decoding of up to 56 bits using bmi2 instructions.
/// Reads an uleb128 encoded integer with at most 56 bits (8 bytes with 7 bits worth of payload each).
/// Returns the integer and the number of bytes that made up this integer.
/// If the returned length is bigger than 8 this means the integer required more than 8 bytes and the remaining bytes need to be read sequentially and combined with the return value.
/// Safety: `data` needs to contain at least 8 bytes.
#[target_feature(enable="bmi2")]
pub unsafe fn decode_uleb_bmi2(data: &[u8]) -> (u64, usize) {
    const CONT_MARKER: u64 = 0x80808080_80808080;
    debug_assert!(data.len() >= 8);

    unsafe {
        let word = data.as_ptr().cast::<u64>().read_unaligned();
        // mask indicating continuation bytes
        let mask = std::arch::x86_64::_pext_u64(word, CONT_MARKER);
        let len = (!mask).trailing_zeros() + 1;
        // which payload bits to extract
        let ext = std::arch::x86_64::_bzhi_u64(!CONT_MARKER, 8*len);
        let payload = std::arch::x86_64::_pext_u64(word, ext);

        (payload, len as _)
    }
}

Branchless uleb encoding for the writing side (not yet tested):
const REQUIRED_BYTES: [u8; 33] = [
    1, 1, 1, 1, 1, 1, 1,
    2, 2, 2, 2, 2, 2, 2,
    3, 3, 3, 3, 3, 3, 3,
    4, 4, 4, 4, 4, 4, 4,
    5, 5, 5, 5, 5
];

/// Safety: `data` needs to have space for at least 8 bytes.
#[target_feature(enable="bmi2")]
pub unsafe fn encode_uleb_bmi2(value: u32, output: &mut [u8]) -> usize {
    const CONT_MARKER: u64 = 0x80808080_80808080;
    const EXPAND_MASK: u64 = 0x7F7F7F_7F7F7F7F;
    unsafe {
        let required_bits = 32-value.leading_zeros();

        //let required_bytes = required_bits/7 + 1;
        let required_bytes = REQUIRED_BYTES[required_bits as usize];
        let expanded = std::arch::x86_64::_pdep_u64(value as u64, EXPAND_MASK);
        let marker = std::arch::x86_64::_bzhi_u64(CONT_MARKER, required_bytes as u32 * 8);

        let marked = expanded | marker;
        output.get_unchecked_mut(0..8).copy_from_slice(&marked.to_le_bytes());
        required_bytes as _
    }
}


Switch endpoint - use timeout for connect

Connect does not return clean on error, and hangs forever
Use a 30 second timeout on switch endpoint.

Done


Added Light mode for the website

Fixes Issue🛠
Closes #515
Description👨💻
Added the changes needed for introducing the light mode for the website. Introduced the toggle switch in the navbar
Type of change📄

[X] New feature (Light mode for the website)

Checklist✅

[X] My code follows the style guidelines of this project
[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[X] I am an Open Source Contributor

Screenshots/GIF📷

Pls work on these:

Just make your changes and no indentation or any other changes in index.html file or any of the files.

Use some good animation or better theme toggling button.

Update the colors in the light theme of this text.

Modify the text colors as it is conflicting with the white background and not visible properly!!
@Yuvraj960

Any update on this PR @Yuvraj960

Work going on @Rakesh9100 , actually time gone for choosing the perfect colors for the font, found that. Now implementing on whole website. And for the animation of the toggle, I am looking for a JS file which will do this work for us. Will attach the screenshots tomorrow showing the progress of the Light mode in the PR

Okay, carry on @Yuvraj960

@Rakesh9100, please view the changes.
I have modified the code in accordance with changes stated by you above.

The animation which is added is lagging as you can also check the deploy preview.
Pls implement a good toggling button as mentioned before.

First mentioned change is not fixed as still unnecessary indentation changes are shown, if you are unable to then you can leave it and I will fix it. @Yuvraj960

@Rakesh9100, first of all very sorry for the indentation issue. I didn't know how to resolve it.
And secondly my animation is not lagging as I have checked in the deploy preview also:
https://github.com/Rakesh9100/CalcDiverse/assets/146326636/8c2064ec-ab49-4a5b-8aa0-33153379c68a
And can you please elaborate me, that what do you meant by the "good toggling button"? I am still stuck in this.

Okay, leave the indentation part. I will see it.
Animation change is lagging sometimes not every time.
Change the toggling button means implement some buttons like for examples:

Check this sun and moon toggling design here.
You can check this.
Check different toggling design effects here.

You can go with any one of these or any similar kind of button instead of using a simple toggle button. Hope this clears your doubt!! @Yuvraj960

@Rakesh9100, I have made the new switch button, kindly check its implementation. I hope with the most recent commit I have done in the button, you like this type of presentation now.

Remove the animation of the website when changing the theme. I am not talking about the button, but the animation. Theme should change immediate when button is clicked.
Reduce the size of the button slightly.
This button should be present in every page navbars such as Contributors, FAQ, and Contact.
Once we change the theme let's say in home page, it should change the theme for the entirely website for every pages.

@Yuvraj960

Ok @Rakesh9100 , I understand all changes you need in the website, as there are many changes to be done in the website, It will take time, but I ensure you, I will give my best.
Kindly assign me one of my another issues in this project, as I have to work on them also along with this issue in progress.

Sure @Yuvraj960

Are you working @Yuvraj960

Yes @Rakesh9100 I am working, due to my weekend exam,  I didn't replied

@Rakesh9100, I need some help in this issue. Actually I am unable to understand the structure of the website.
I have implemented the first 2 changes you demanded.
But while adding the button to all the navbars, I am encountering an unknown issue.
The issue is that navbar size is not fixed, it varies on different pages. I came to know about this issue when I added the dark-light switch button in all the navbars. The margins and paddings are not uniform through all website navbars.
While I was fixing the size of my button, the rest navbar button just left shift to an arbitrary place.
Can you please to some Favour for me.
Can you please open a separate issue for matching the light mode for the whole website,
And I merge the PR for the topic under Home-Page_Dark-Light_Theme-Design?
I am saying this only because I am unable to understand the file structure of the website, that which element of the website is contributing in which way to the pages of the website.
I can design the light mode for all the pages and write the code for them also, only problem is with linking of those pages, that I am currently confused in, how to do it.

@Rakesh9100 , please address me regarding this issue.


Replacing ScanConfig default with Scanconfig::new(true) wherever possible as in most of the cases we do not need sorted results

Problem
Currently ScanConfig::default sets collect_unsorted to false which has a huge performance impact.
We spend too much time to sort the results when we really do not need to sort the results.
This change will improve performance by x2-x3 for large results.
Summary of Changes
Instead of changing default, i removed the default and set ScanConfig::new(true) to ensure that we do not add any regessions.
Fixes #

@CriesofCarrots Can you take a look too, please? Do you know if the CLI/RPC output changes will impact user expectations?

I see the usages in rpc.rs have already been updated to use ScanConfig::new(!RpcProgramAccountsConfig::sort_results). It looks like the non_circulating_supply results in rpc could change, as well as the results in solana_rpc_client (due to usage of RpcProgramAccountsConfig::default()). But we don't make any guarantees about ordering in RPC responses with account scan results, so I think this is okay.
The one place we might want to preserve existing behavior is in agave_ledger_tool. While I assume most engineers are not making assumptions about account order, it might be a little annoying for the default to change, or at least have no way to get sorted results (especially for TestValidator usage where the number of accounts is small). Could we preserve sorted in that bin in this PR? We could add cli configuration for --sorted and change the default in a follow-up.

@CriesofCarrots Can you take a look too, please? Do you know if the CLI/RPC output changes will impact user expectations?

I see the usages in rpc.rs have already been updated to use ScanConfig::new(!RpcProgramAccountsConfig::sort_results). It looks like the non_circulating_supply results in rpc could change, as well as the results in solana_rpc_client (due to usage of RpcProgramAccountsConfig::default()). But we don't make any guarantees about ordering in RPC responses with account scan results, so I think this is okay.
The one place we might want to preserve existing behavior is in agave_ledger_tool. While I assume most engineers are not making assumptions about account order, it might be a little annoying for the default to change, or at least have no way to get sorted results (especially for TestValidator usage where the number of accounts is small). Could we preserve sorted in that bin in this PR? We could add cli configuration for --sorted and change the default in a follow-up.

Ok for now by default all ledger tool will use sorted : true. Will add the option in another PR.
Restoring the ScanConfig::Default with value for collect_unsorted = true.
Minor change adding sort_results for get_large_accounts method.

I had already updated but may be there is standard of naming PR in solana which i am unware of.

Merge at your preference.


Merge Sorted Array

hey, can i take this issue up?

hey @vidhichadha2507, I want to contribute to this issue. can you please assign this to me?

hey, can i take this issue up?

Sure... You can work on this issue now

hey, can i take this issue up?

Sure... You can work on this issue now

Have you completed?

Can I take up this issue?
🐧will try my best to respond within a day

Hey @vidhichadha2507  I want to add an efficient approach to this issue . Can you please assign this to me.


Add Pickles_types.Higher_kinded_poly

This PR adds the Pickles_types.Higher_kinded_poly utility library.
This is a type-level hack that allows manipulating higher-kinded polymorphic types using type variables.
For example, we can constrain packed modules to have a particular higher-kinded type even though we cannot do so in the surface-level type system:
module My_type = struct
  type 'a t
end
module type S = sig
  type 'a t
  module Poly_witness : P1.S with type 'a p = 'a t
end
let f (module M : S with type Poly_witness.witness = P1.W(My_type).t) = ...
let g (type t_poly) (module M : S with type Poly_witness.witness = t_poly) (module N : S with type Poly_witness.witness = t_poly) = ...

This also plays nicely with Hlist. For our purposes, we can use this to carry proofs from multiple proof systems in the same Hlist:
let my_list_of_lists_of_proofs : ('varss, 'valuess, 'num_parentsss, 'num_rulesss, 'proof_systems) H4_1.T(P4) =
  [ [PS1.to_poly proof1; PS1.to_poly proof2; PS1.to_poly proof3]
  ; [PS2.to_poly proof4; PS2.to_poly proof5] ]
(* Actual type is *)
val my_list_of_lists_of_proofs :
   ( ('var1 * ('var2 * ('var3 * unit))) * (('var4 * ('var5 * unit)) * unit)
   , ('value1 * ('value2 * ('value3 * unit))) * (('value4 * ('value5 * unit)) * unit)
   , ('num_parents1 * ('num_parents2 * ('num_parents3 * unit))) * (('num_parents4 * ('num_parents5 * unit) * unit)
   , ('num_rules1 * ('num_rules2 * ('num_rules3 * unit))) * (('num_rules4 * ('num_rules5 * unit) * unit)
   , P4.W(PS1.T).t * (P4.W(PS2.t * unit) )
   H4_1.T(P4)

and then the prover can operate over a defined set of proof systems (P4.W(PS1.T).t * ((P4.W(PS2.T).t * unit)) H1(Proof_system).t that it accepts as a parameter.

As a (hopefully) compelling argument for the safety of this, my first version of this used a (nearly) equivalent formulation:
module P1 : sig (* snip *) end = struct
  type (_, _) t = ..
  module T (M : T1) : sig (* snip *) end = struct
    type (_, _) t += T : 'a M.t -> ('a, O.t1 M.t) t
    let to_poly x = T x
    (* Equivalent to [match x with T x -> x], except we don't check the tag of the constructor,
       so multiple functor calls that result in different functors will still alias if the type
       is the same.
    *)
    let of_poly x = Obj.(obj (field (repr x) 1))
  end
end

This is necessarily safe, as the only low-level manipulation is to bypass the tag equality check from the match, and the field access otherwise matches what match would do. This current version is nearly functionally equivalent, except avoiding the wrapping constructor lets us write more-concise, more efficient code by coercing directly and avoiding the wrapping allocations.
Checklist:

[ ] Document code purpose, how to use it

Mention expected invariants, implicit constraints

[ ] Tests were added for the new behavior

Document test purpose, significance of failures
Test names should reflect their purpose

[ ] All tests pass (CI will check this if you didn't)
[ ] Serialized types are in stable-versioned modules
[ ] Does this close issues? List them:

😎

Do you think it is worth considering either of

The equality coercion here saves us quite a lot of wrapping and unwrapping. For example, transforming a ((a, b, c, poly_var), (x, y, z, poly_value)) Typ.t requires quite a lot of boilerplate in both of those libraries, but with this we can just do a let T = (Path.To.mk_eq () : ...) in. I also felt that having explicit distinct types parameters played more nicely with Hlist.
Otherwise, the underlying idea is the same. No such thing as an original thought, I guess 😁


[FuseConsumer] Update consumer fusion pass to be more generic

-- This commit updates consumer fusion pass to be more generic whilst trying to pick the compute op/loop to fuse with.
-- The idea is to hook on to the innermost SCF loop -> find a compute op within it -> try to aggressively fuse the consumer.
-- As a result it removes useSCFFor pass flag.
-- Helps in increasing L2 tile size task.
Signed-off-by: Abhishek Varma<EMAIL_ADDRESS>Before : IR dump - (refer this e2e IR chain)
After this fix : IR dump

Merging this in as this is anyway an improvement into the existing functionality of the pass.


update to pkg/browser to open URLs

Signed-off-by: Navendu Pottekkat<EMAIL_ADDRESS>Description
This PR fixes #3664
Updated to pkg/browser to open URLs in mesheryctl.
Notes for Reviewers
Please test out this PR on multiple OSes by running mesheryctl system start and checking if Meshery is being opened in the browser.

[x] MacOS/M1

[ ] MacOS/Intel

[x] Linux

[ ] Windows

[ ] Yes, I signed my commits.

Codecov Report

Merging #3682 (ff4d686) into master (47b48f9) will decrease coverage by 7.44%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master    #3682      +/-   ##
==========================================
- Coverage   20.35%   12.90%   -7.45%
==========================================
  Files          45       45
  Lines        3788     3773      -15
==========================================
- Hits          771      487     -284
- Misses       2812     3171     +359
+ Partials      205      115      -90

Flag
Coverage Δ

gointegrationtests
?

unittests
12.90% <0.00%> (+0.05%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

mesheryctl/pkg/utils/helpers.go
18.75% <0.00%> (+0.80%)
:arrow_up:

mesheryctl/internal/cli/root/system/status.go
0.00% <0.00%> (-62.83%)
:arrow_down:

mesheryctl/internal/cli/root/system/start.go
1.18% <0.00%> (-51.58%)
:arrow_down:

mesheryctl/internal/cli/root/system/stop.go
0.90% <0.00%> (-39.10%)
:arrow_down:

mesheryctl/internal/cli/root/system/check.go
1.48% <0.00%> (-22.68%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 236c13c...ff4d686. Read the comment docs.

@piyushsingariya any idea why the test fails?

LGTM. My test on MacOS/Intel was successful.

@leecalcote The workflow should not have failed, I am checking to see if we can fix it. It says it is because of the missing xdg-open.
@piyushsingariya were you able to draw any better insight?


Use bignumber.js (or similar) for fee calculations and dealer

javascript is not good with big numbers calcs
refs:

well-known issue
https://itnext.io/lets-talk-about-big-numbers-in-javascript-b0a2c8edb170
https://github.com/MikeMcl/bignumber.js/

I think we could just used the internal bigNum library of javascript now. don't need an external dependancy.
related: https://github.com/GaloyMoney/galoy/issues/39

maybe with integers we have no problems but feeRatio has decimals, so this need to be reviewed carefully

even with integer this would be useful to have bigint I think. it's not possible to have 21m milli sats in integer for instance.
I started working on a PR for this multiple months ago but it as I was going it was starting to be a bigger refactor than I anticipated so I moved on to some more urgent things


get examples mopre detailed

Hi!
The project is awesome.
Could you post the app code running at http://meteoric.github.io/ for better comprehention?
(just learning ionic and would help me a lot to see the working examples)
Thanks a lot!

https://github.com/meteoric/demo


save_pretrained() creates incorrect adapter_model.bin file, when training LoRA model with multi-card zero-3

System Info
peft=0.5.0
accelerate=0.23.0
transformers=4.33.2
torch=2.0.1
python=3.8.18
Who can help?
No response
Information

[ ] The official example scripts
[X] My own modified scripts

Tasks

[ ] An officially supported task in the examples folder
[X] My own task or dataset (give details below)

Reproduction
main.py:
import torch

from torch.utils.data.dataloader import DataLoader
from torch.utils.data.dataset import Dataset

from datasets import load_dataset

import transformers
from accelerate import Accelerator
from tqdm import tqdm

import peft

class myset(Dataset):
    ...

def main():
    accelerator = Accelerator()
    model = transformers.LlamaForCausalLM.from_pretrained("******/zh-llama/7b")
    loraconfig = peft.LoraConfig(
        r=64,
        lora_alpha=128,
        lora_dropout=0.05,
        target_modules=[
            "q_proj",
            "v_proj",
            "k_proj",
#            "o_proj",
#            "gate_proj",
#            "down_proj",
#            "up_proj",
        ],
        task_type=peft.TaskType.CAUSAL_LM,
    )
    model = peft.PeftModel(model, loraconfig)
    model.print_trainable_parameters()
    tokenizer = transformers.LlamaTokenizer.from_pretrained("******/zh-llama/tokenizer")
    optimizer = torch.optim.AdamW(model.parameters(), lr = 1e-5, weight_decay = 1e-5)
    #optimizer = DummyOptim(model.parameters(), fp32_optimizer_states=False)
    #scheduler = DummyScheduler(optimizer, 5850 * 3, 5850 * 3 / 10)
    trainset = myset(
        load_dataset(
            "/home/hos/workspace/datasets/madao33/new-title-chinese",
            split="train",
        ),
        tokenizer,
    )
    trainloader = DataLoader(
        trainset,
        batch_size=1,
        shuffle=True,
        num_workers=1,
        collate_fn=transformers.default_data_collator,
    )
    # testloader = DataLoader(testset, batch_size=2, shuffle=False, num_workers=4)
    model, optimizer, trainloader = accelerator.prepare(
        model, optimizer, trainloader
    )
    print(model)
    model.train()
    #deepspeed.DeepSpeedEngine
    accelerator.wait_for_everyone()
    for epoch in range(3):
        totalloss = 0.0
        for step, batch in enumerate(tqdm(trainloader)):
            if step > 10:
                break
            for key in batch.keys():
                batch[key] = batch[key].cuda()
            loss:torch.Tensor = model(
                input_ids=batch["input_ids"],
                attention_mask=batch["attention_mask"],
                labels=batch["labels"],
            ).loss
            totalloss += loss.mean().item()
            accelerator.backward(loss)
            optimizer.step()
            #scheduler.step()
            optimizer.zero_grad()
            accelerator.wait_for_everyone()

        print("Epoch ", epoch, " avg loss: ", totalloss/len(trainloader))
        #unwrapped_model:peft.PeftModel = accelerator.unwrap_model(model)
        accelerator.wait_for_everyone()
        model.save_pretrained(
            f"./checkpoints/epoch-{epoch}",
            is_main_process=accelerator.is_main_process,
            save_function=accelerator.save,
            state_dict=accelerator.get_state_dict(model),
        )
        accelerator.wait_for_everyone()

if __name__ == "__main__":
    main()

ds.yaml:
compute_environment: LOCAL_MACHINE
deepspeed_config:
  gradient_accumulation_steps: 1
  gradient_clipping: 1.0
  offload_optimizer_device: 'cpu'
  offload_param_device: 'cpu'
  zero3_init_flag: true
  zero3_save_16bit_model: true
  zero_stage: 3
  contiguous_gradients: true
distributed_type: DEEPSPEED
downcast_bf16: 'no'
dynamo_backend: 'NO'
fsdp_config: {}
machine_rank: 0
main_training_function: main
megatron_lm_config: {}
mixed_precision: 'no'
num_machines: 1
num_processes: 2
rdzv_backend: static
same_network: true
use_cpu: false

train.sh:
#!/bin/sh
export OMP_NUM_THREADS=8
accelerate launch --config_file ds.yaml main.py

I tried every method of saving models, including using unwrap_model(model) and adjusting parameters of save_pretrained() function, etc, but none of them works.
Expected behavior
It seemed OK when the training script ends, but when I tryed to load the model, it says "PytorchStreamReader failed reading file data/0":
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/hos/miniconda3/envs/pyend/lib/python3.8/site-packages/transformers/modeling_utils.py", line 3260, in from_pretrained
    model.load_adapter(
  File "/home/hos/miniconda3/envs/pyend/lib/python3.8/site-packages/transformers/integrations/peft.py", line 151, in load_adapter
    adapter_state_dict = load_peft_weights(peft_model_id, revision=revision, use_auth_token=token)
  File "/home/hos/miniconda3/envs/pyend/lib/python3.8/site-packages/peft/utils/save_and_load.py", line 198, in load_peft_weights
    adapters_weights = torch.load(filename, map_location=torch.device(device))
  File "/home/hos/miniconda3/envs/pyend/lib/python3.8/site-packages/torch/serialization.py", line 809, in load
    return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
  File "/home/hos/miniconda3/envs/pyend/lib/python3.8/site-packages/torch/serialization.py", line 1172, in _load
    result = unpickler.load()
  File "/home/hos/miniconda3/envs/pyend/lib/python3.8/site-packages/torch/serialization.py", line 1142, in persistent_load
    typed_storage = load_tensor(dtype, nbytes, key, _maybe_decode_ascii(location))
  File "/home/hos/miniconda3/envs/pyend/lib/python3.8/site-packages/torch/serialization.py", line 1112, in load_tensor
    storage = zip_file.get_storage_from_record(name, numel, torch.UntypedStorage)._typed_storage()._untyped_storage
RuntimeError: PytorchStreamReader failed reading file data/0: invalid header or archive is corrupted

I tried unzip adapter_model.bin, discovering that the first few packages were missing or broken.
(base) ***@*****:*/checkpoints/epoch-1$ unzip adapter_model.bin
Archive:  adapter_model.bin
 extracting: adapter_model/data.pkl   bad CRC 7d367a1c  (should be 3b14ea4d)
file #2:  bad zipfile offset (local header sig):  28738
file #3:  bad zipfile offset (local header sig):  1077392
file #4:  bad zipfile offset (local header sig):  2126096
file #5:  bad zipfile offset (local header sig):  3174800
 extracting: adapter_model/data/101
 extracting: adapter_model/data/102
 extracting: adapter_model/data/103
 extracting: adapter_model/data/104
 extracting: adapter_model/data/105
 extracting: adapter_model/data/106
 extracting: adapter_model/data/107
 extracting: adapter_model/data/108
 extracting: adapter_model/data/109
 extracting: adapter_model/data/11
 extracting: adapter_model/data/110
 extracting: adapter_model/data/111
...

Sometimes this bug doesn't appear when tested with tiny models. Generally the larger the model is, the more likely it will appear.

@pacman100 @younesbelkada
Please see this issue, thank you!

This seems to be the same issue as in https://github.com/huggingface/accelerate/issues/1922.
I wonder this PR might fix the problem, would it be possible for you to test again using that accelerate branch?

Do you mean that we should add a save_function argument to save_pretrained, so as to be able to pass accelerator.save? Yes, I think that would make sense.


[compatibility] player join/quit/death event

Hello!
When the Chatty plugin is installed, all messages about the player’s join/quit/death, including messages from other plugins (for example, DeathMessagesPrime) are blocked if
misc:
  join/quit/death:
    message: ''

or overwritten if
misc:
  join/quit/death:
    message: 'anything'

I ask you to add an option to the config, which would allow you to completely disable listening to the join/quit/death events.
Best regards.

Hello!
You can just remove "join/quit/death" block from "misc" section and events won't be blocked.
For example, if you need change "join" and "quit" messages, but don't want block death messages, you can write this lines (without death block):
misc:   join:     message: '&e{prefix}{player}{suffix} &ejoined the server.'     permission: false   quit:     permission: false

Thanks a lot! It working now!


Multus with Contiv

Hi, I was wondering, since you've mentioned Contiv as one of the CNI plugins you support, has anyone done any real work integrating the two? I'm struggling to understand how to pass network names to contiv through CNI configuration in NetworkAttachmentDefinition, since by default they try to read them from pod labels.

@networkop Could you please explain the details. FYI https://github.com/contiv/vpp/tree/master/cmd/contiv-cni

Hi @rkamudhan , sure.
I'm trying to get contiv working with multus. Specifically, I want to attach a pod to a specific network inside contiv. So normally, without multus, I do that by specifying labels inside pod's metadata, e.g. if i want to attach my pod to contiv-net:
apiVersion: v1
kind: Pod
metadata:
  labels:
    io.contiv.network: contiv-net

What I want is to use multus to attach a pod to multiple contiv networks, which would have been possible if I could specify io.contiv.network inside a NetworkAttachmentDefinition.
But now I've had a closer look at the contiv cni plugin and it looks like they're taking those label values by quering api server directly and not from a cni configuration file, so doesn't look like it's possible.

I have the same question: how to integrate Multus with Contiv correctly?
Did @networkop solve it successfully, so closed this issue?
Would you share the way you get it works, thank you!

I'm interested in integrating contiv with multus as well. @networkop didn't solved it as it didn't seem possible at the time. But that was in 2018. Maybe things changed since then.


Check for updates showing error!

Describe the bug or Issue
The system cannot find the file specified
File: D:\Portable
Arguments: Software\StaxRip\Update.bat
Step To Reproduce the Problem

Go to 'Advanced'
Click on 'Check for update'

Screenshots
https://imgur.com/a/02b4ijG
Desktop (please complete the following information):

OS: [Windows 10 Build 1803]
Version [2.0]

Additional context
I updated to latest 2.0 but I still get this error

It keeps running into TLS issue & it seems you might have PowerShell security in locked stage preventing from certain functions to execute.

This way I can also use much higher level functions and I don't have to deal with PowerShell dumb coding.


Accept array in Headers constructor

From the spec, it looks like the Headers constructor should accept an array of arrays.
I've adapted the example for the tests:
var meta = { "Content-Type": "text/xml", "Breaking-Bad": "<3" }
new Headers(meta)

// The above is equivalent to
var meta = [
  [ "Content-Type", "text/xml" ],
  [ "Breaking-Bad", "<3" ]
]
new Headers(meta)

This came up for me as an issue using stream-http in Edge with the Financial Times polyfill.  I've opened jhiesey/stream-http#71 and Financial-Times/polyfill-service#1091 in case this can be addressed elsewhere, but it looks to me like this might be the right fix.

The native Headers implementation in Chrome, Firefox, and Edge accept an array in the constructor. IE 11 and Safari 10, where this polyfill is active, does not, so this is indeed a bug in the Headers polyfill.
Thanks for the detailed bug report and fix.

Thanks for pulling this in @dgraham.  Any chance for an updated release with the fix?

The 2.0.3 release is now available on npm.


optimize pandas queries for visualizations

Augur queries are bloated, requesting lots of unused data and storing strings for integer values naively. Clean these up.

part of the Redis-stabilization milestone

[ ] switch from CSV to Feather format - look into how to shard large repos
[ ] more data preprocessing pre-storage.

@JamesKunstle can this be closed?

@cdolfi no still working on this.

checked total data size. optimizations at this point aren't necessary- app runs fine w/o further optimization, we can focus on optimizations when we're further along.
possible opts include:

dropping unused data from queries that no callback is using
cast large integers down and repeated strings to categories
finally finish csv->feather format if necessary.


Adding base_namespace option support to Grpc.Tools

I added support in two ways and kept the old behavior as default to not break any users. It's also more efficient if Protobuf.BaseNamespaceEnabled is kept on false as it requires looking into the file content to find out the correct path. Additionally to the attribute, I added a property Protobuf_BaseNamespaceEnabledDefault to enable it globally (in case a lot of entries would have to be enabled).
Blocked by #22627
Fixes #17672
/CC: @jtattermusch @kkm000

We should make sure that #22628 is kept in documentation when this is merged as it's more performant and working for most uses where relative paths are used. This solution is working in all use cases like absolute use cases and therefore still needed.

While I understand that there might be some use for the base_namespace option when generating from the .proto files manually (or with a script), I think using base_namespace together with Grpc.Tools base_namespace support as a solution for https://github.com/grpc/grpc/issues/17672 is an overkill.
When using Grpc.Tools, you actually don't care about the exact path of the .cs files under obj, as long as they are unique (= 2 helloworld.proto files in different places need to generate 2 different .cs files under obj/Config/). So choosing a solution where you first introduce an option that generates namespace name based on C#-specific name mangling rules, and then try to get Grpc.Tools to mimic that mangled path (which requires analyzing file content with a regex in a very fragile way) sounds like overcomplicating things. Instead we should come up with some simple way of making the generated .cs file paths under obj/Config unique (I still believe this should be doable when the right trick is used), rather than bending ourselves backward trying to fix this with base_namespace and adding a lot of new complexity only to come up with a solution that is actually quite fragile.

Will be superseded by the approach from here: https://github.com/grpc/grpc/issues/17672#issuecomment-621152800


Document missing information on excluding SharePoint sites from Retention Policy

There is the option in Retention Policy under SharePoint to exclude specified sites from the policy. This capability is not mentioned or detailed in this document. Getting this feature to work is problematic at best.
Please create some detailed instructions on how to exclude a SharePoint site from the policy. Also include information and steps on how to investigate known errors (eg when exclusions don't appear to apply)

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 36b54827-f642-4c77-7cfe-64e8c2a1d1d9
Version Independent ID: 67dffecf-4da7-864d-5c10-9b55740f32cd
Content: Learn about retention for SharePoint and OneDrive - Microsoft 365 Compliance
Content Source: microsoft-365/compliance/retention-policies-sharepoint.md
Service: o365-seccomp
GitHub Login: @cabailey
Microsoft Alias: cabailey

@Murdos-AU  Thank you for your feedback.  This page focuses to how retention works for SharePoint and OneDrive.  For instructions to configure a retention policy, see https://docs.microsoft.com/en-us/microsoft-365/compliance/create-retention-policies?view=o365-worldwide.
Specifically for configuring the SharePoint and OneDrive locations, see https://docs.microsoft.com/en-us/microsoft-365/compliance/create-retention-policies?view=o365-worldwide#configuration-information-for-sharepoint-sites-and-onedrive-accounts
I'm not aware of problems for excluding sites - it should be a case of selecting "Exclude sites" and then providing the site URL, which is validated when you complete the wizard.  If you are excluding a site and items are still be retained/deleted in that site, and these items are not subject to another retention policy or retention label, I suggest you open a ticket with Support for investigation.  If I hear of any known issues, I'll update this thread.


Add new contributor & post

Description of changes:
Added Aggelos Arvanitakis as a contributor, along with the post titled "Creating reusable abstractions with Amplify and React hooks"
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@3nvi Welcome to the community.
We recreated your PR here as we accidentally deleted the gitignore file. Closing PR and referencing: https://github.com/aws-amplify/community/pull/90


Following the instructions did not yield a service I could actually start. I had to invoke UseWindowsService on the IHostBuilder in Program.CreateHostBuilder.

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ed148088-4550-2788-0756-390eab15e3f6
Version Independent ID: 64d5cdf8-e50f-0c89-7f16-82645ba6383e
Content: Create a Windows Service using BackgroundService
Content Source: docs/core/extensions/windows-service.md
Product: dotnet-fundamentals
GitHub Login: @IEvangelist
Microsoft Alias: dapine

Hi @stevenwbass - I'm sorry that you had a bad experience, but this step is right there in the article. You must have skimmed past this or something?

To help improve its visibility I'm adding another heading to help with discoverability. Instead of being nested under the Rewrite the Worker class heading, it will be under a new heading called Rewrite the Program class.

Ah, yes, I must have. Thank you!


[Impeller] validation errors on debug builds of AiksTest.CanDrawPoints in impeller_golden_tests

I believe validation layers are used for used for CI.  Maybe this validation error only happens with debug builds?
see in
6cefbe183546461bdcdf7633a4cd5021cce2f210
report
--- Vulkan Debug Report  ----------------------------------------
|                Severity: Error
|                    Type: { Validation }
|                 ID Name: VUID-VkWriteDescriptorSet-descriptorType-00328
|               ID Number: -368569266
|       Queue Breadcrumbs: [NONE]
|  CMD Buffer Breadcrumbs: [NONE]
|         Related Objects: Device [5234753552] [ImpellerDevice]
|                 Trigger: Validation Error: [ VUID-VkWriteDescriptorSet-descriptorType-00328 ] Object 0: handle = 0x138040010, name = ImpellerDevice, type = VK_OBJECT_TYPE_DEVICE; | MessageID = 0xea08144e | vkUpdateDescriptorSets(): pDescriptorWrites[1].pBufferInfo[0].offset (0x38) must be a multiple of device limit minStorageBufferOffsetAlignment 0x100. The Vulkan spec states: If descriptorType is VK_DESCRIPTOR_TYPE_STORAGE_BUFFER or VK_DESCRIPTOR_TYPE_STORAGE_BUFFER_DYNAMIC, the offset member of each element of pBufferInfo must be a multiple of VkPhysicalDeviceLimits::minStorageBufferOffsetAlignment (https://vulkan.lunarg.com/doc/view/<IP_ADDRESS>/mac/1.3-extensions/vkspec.html#VUID-VkWriteDescriptorSet-descriptorType-00328)
-----------------------------------------------------------------

Fixed in https://github.com/flutter/engine/pull/51341


refactor: remove voting functionality

Closes #434
Closes #427

Let's instead try to improve on vote functionality for now.


Valid location derivation of patient to show question paper

currently questionnaire is localised by country and zippin code of user given at time of registration, but we may change it to get pincode from current (quarantine) location and find pincode from that and then give questionnaire based on that


Publish to Crates?

Would you be willing to publish this to Crates?
https://crates.io/search?q=hipchat-client

Short answer:
Yes, if this is useful, publishing to crates.io can be done.
Long answer:
I had not published this yet because it is very much a work in-progress.
I'm currently running into situations where the JSON in the HipChat API contains undocumented properties, which will in turn result in tripping over serde-rs/serde/issues/44.
This can be replicated by running the integration tests. I get the following error:
thread 'integration_get_room' panicked at 'called `Result::unwrap()` on an `Err` value: Json(SyntaxError(unknown field "last_active", 1, 162))', ../src/libcore/result.rs:688

Even though there is no mention of last_active in the HipChat API documentation: https://www.hipchat.com/docs/apiv2/method/get_room.
...maybe I should just switch back to rustc_serialize for now :disappointed:.

Hey @rsolomo - thanks for the detailed response during the holidays!
Frustrating about the additional properties - selfishly, I know that this library would be useful on Crates in its current form for me, as I want to use it primarily for posting notifications to rooms.
Anyway, will await your decision :mortar_board:


[starter-kit] A targeted folder for build artifacts

Should this be an RFC?

[X] This is not a substantial change

Which package is this a feature request for?
Lit Core (lit / lit-html / lit-element / reactive-element)
Description
It would be nice if all the built lit resources ended up in a target folder.  This makes it easier to change the project.json scripts to accomodate changing the element name and also building more than one element within a single lit project.
The docs part of lit takes this approach.  This is how the docs:clean script looks.
"docs:clean": "rimraf docs",

Alternatives and Workarounds
The clean scripts deletes built lit resources like this.
 "clean": "rimraf my-element.{d.ts,d.ts.map,js,js.map} test/my-element.{d.ts,d.ts.map,js,js.map} test/my-element_test.{d.ts,d.ts.map,js,js.map}",

And so if we add more elements or change the name my-element we have to modify these scripts.
If the built resources ended up in for example target/main and target/test, then we could just have the clean script delete the content of these folders and it keeps the root folder less "Messy" which I think is an "Aesthetically" pleasing thing ...

I think this issue is referring to starter kits, which are currently tailored for authoring and publishing a single component.
We have wanted for a long time to show case different set ups for also authoring a design system (multiple components in a package with interdependencies and styles), and a whole app too (https://github.com/lit/lit/issues/1929). But project starters end up being very opinionated hard to come up with a one solution fits all. In the end, the user will have to modify the starter kit to fit their use case anyway.
I will add that the reason why we emit the the built artifact to the root directory which could make it look "messy" is that older build tools/bundlers with outdated module resolution works best when the artifact locations in the published package match the hierarchy particularly for sub-modules like if you want to make import 'my-package/components/my-component.js' work. These days it's less of a concern with package exports being more ubiquitous.

@augustjk if I remember correctly the publish script for lit publishes all the contents of the root directory.
One idea would be to built a publish directory with all the resources that need to be published.
This way the published directory could also be cleaned with a simple script.

Another thought is ... Currently the checksize script looks like this:
    "checksize": "rollup -c ; cat fs-link-preview.bundled.js | gzip -9 | wc -c ; rm fs-link-preview.bundled.js"

It would be nice if it was something more generic like this:
"checksize": "rollup -c ; cat index.js | gzip -9 | wc -c ; rm index.bundled.js"

And part of this is I was wondering whether the thing that gets published and referenced by external libraries / browsers could index.js that is namespaced under a package directory.
This way the bundle stays generically named, and this would allow for generic named components to be broken out and be independent of the project name space essentially ...

The .gitignore files can also be simplified since it can just ignore a folder instead of top level resources like this.
# top level source
my-element.js
my-element.js.map
my-element.d.ts
my-element.d.ts.map

I will add that the reason why we emit the the built artifact to the root directory which could make it look "messy" is that older build tools/bundlers with outdated module resolution works best when the artifact locations in the published package match the hierarchy particularly for sub-modules like if you want to make import 'my-package/components/my-component.js' work. These days it's less of a concern with package exports being more ubiquitous.

I want to reiterate that this is why we output top-level build artifacts. Many people did want to make simple imports like:
import 'my-package/components/my-component.js';

It's a pretty fundamental thing with starter kits that they encode a lot of opinions, and that very reasonable people might disagree on those opinions. This is one of those things 🤷
Now maybe package exports are widely supported enough now that we could recommend having a package file layout that differs from the exports layout. That would solve this.

Now maybe package exports are widely supported enough now that we could recommend having a package file layout that differs from the exports layout. That would solve this.

I think if we create a publish directory for all resources published, then ultimately we will end up basically with what we have now.  The semantics are all the same with a few exceptions.

We copy all resources that are going to be published into the publish directory.  So the publish directory is now essentially the new root directory.
instead of having a specifically named file my-element.ts as a starting point, instead it's called index.ts.  That's the starting point.  And all resources that are going to be published are imported into index.ts.

This way users do not need to rename anything, if they want to change the file name of the web component.

Here's how I think the entire thing could be done.

Compile so that all built resources are placed in a build directory.
Refactor package.json export properties like this:

  "main": "index.bundle.js",
  "module": "index.js",
  "types": "index.d.ts",
  "type": "module",

Copy all the resources that were being published at the root level into a publish directory.
When publishing cd into the publish directory and just publish the same way that the project was published before.

And the benefits of this are.

Less refactoring of all the files when changing a file name
It's easier to breakout code into separate files, since index.ts is the only client facing resource and all the other files are hidden behind this entry point.


[Question] Documenting "inherited" method

Ruby has a special inherited method that gets called when a class is inherited. How should I document this in Yard? Since this is a hook defined by Ruby and merely implemented by me, I have a feeling I shouldn't document what it does and when, but reference it somewhere else.
Does this look right to you?
# Example parent class.
class Parent
  # @api
  # @see https://ruby-doc.org/core-2.6.3/Class.html#method-i-inherited
  def self.inherited(subclass)
    puts "#{self} inhertied by #{subclass}"
  end
end

# Example child class.
class Child < Parent; end

In a broader perspective the question applies to all Ruby hooks/callbacks like extended, prepended and inherited, but also has some resemblance to constructors like initialize, intialize_copy, initialize_clone and initialize_dup in how the method is invoked by Ruby itself.
Thanks

One thing to remember is that 'hook/callback' methods can be called by any code, so one could say that they are no different than other methods.
One could use @!group directives to indicate that they are different...

It's true the hooks are at a low level methods that can be invoked like any other method, but on a conceptual level they are wildly different. I'm not the one dictating what this method does, what parameters it takes, what it returns etc. I'm merely obeying someone else's API here.

I don't disagree.  You've mentioned some of Ruby Core's hooks/callbacks, but they also exist with RubyGems, OpenSSL, etc.  Gems separate from Ruby also use them.
I suppose a tag like @callback or @hook might be helpful, but one could also use @note, or create their own tag like @note...

Does this look right to you?

This seems overdocumented to me. You certainly don't need to URL link to core documentation. When YARD generates docs with enough global context, it can make the association between callbacks like Class#inherited on its own.
There have been discussions about modeling Ruby core documentation and making it available as a base docset that can be included when core methods are referenced or inherited, but there's a bunch of complexity that this brings and so it hasn't (yet) been implemented. You could do this on a small scale with a plugin or custom hook as noted above, but linking all the way to the docs seems excessive.
As a sidenote, @api takes a parameter. My general suggestion here would be to mark this API as "internal" (@api internal) and have your default docs omit those APIs (using --query<EMAIL_ADDRESS>!= 'internal'") since users of your external APIs don't need to see this. Implementors can generate docs for internals with a different command or you can forego this step if you have a small enough maintainer team and you are all comfortable looking directly at source for these things.
TLDR: the following is sufficient:
# @api internal
def self.inherited(subclass) ... end


Python Extension causing memory/cpu problems

The python extension is causing memory and cpu problems as shown in the image. After executing a fit method on a tensorflow model, the python extension (checked in the process explorer) continued to consume the memory and cpu (attached image). The execution of the code in the .py file was carried out in python interactive mode. Few days before, the memory consumption of the extension continued to increase with time while executing a python file using python interactive mode. Also, I noticed that after execution of any single cell, all the variables in the variable explorer window are reloaded.
Environment data

VS Code version: 1.38.1
Extension version (available under the Extensions sidebar): 2019.10.41019
OS and version: Ubuntu 18.04 LTS
Python version (& distribution if applicable, e.g. Anaconda): 3.6.7
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): virtualenv
Relevant/affected Python packages and their versions:
Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info #3977):

Expected behaviour
CPU and memory usage should be idle after execution
Actual behaviour
But the usage continues to loop

Steps to reproduce:
To be run in python interactive mode

Logs
Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help; turn on source maps to make any tracebacks be useful by running Enable source map support for extension debugging)
XXX

@sameervk Can you provide additional details on which process is taking up memory and cpu? Also, what do you have for this setting:

Jedi or Language Server? (i.e. what is "python.jediEnabled" set to; more info #3977):

@karthiknadig the process in the process explorer window is the extensionHost which is the microsoft python extension.
The python.jediEnabled is set to false. Should I set it to True?

@rchiodo Do you think this could be due to how cells are loaded currently in the extension?

@sameervk is this still and issue.
If so, please could you test the latest insiders build of the extension. We have made a number of improvements in this space.
Instructions can be found here https://github.com/microsoft/vscode-python/blob/master/CONTRIBUTING.md#development-build

@DonJayamanne it seems fine now, thank you.

Thanks for the feedback, please create separate issue(s) for the missing feature, this way we can prioritize them.
Fyi, please do look know existing issues


Specify NodeJS as a required toolset to build this repo

Fails the build early if NodeJS is not available on the machine or is too old.
cref https://github.com/aspnet/BuildTools/pull/448

@natemcmaster,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

Why is this needed? AFAIK This repo still builds fine with nodejs 6. The problem is that it has impact on other projects you might work on on the same machine that were not ported to node 8/npm5.

We need a better handle on which build-dependencies we've accumulated over time. We need to have a complete list of this for the source-build and prodcon build efforts.
Node 6 versus 8: I just picked node 8 because that's what Travis and AppVeyor were configured to install. Plus, Node 8 will become the Node LTS version tomorrow.

@moozzyk would you approve this if I dropped the minVersion to a 6.9 version or is 8.0 okay?

Yes. The required version should be 6.x for now. Travis/Appveyor should continue to use 8. Once version 8 become more common (I don't actually think we have 8 on our TC agents) we can make 8 required. For a context - I needed to submit a PR to node project and had to keep deleting package-lock.json file because they were running 6. The installation is global so once you have 8 it affects all the repos.

Ok. Updated

AppVeyor failed with an unrelated error. Is this a known flaky test?
icrosoft.AspNetCore.SignalR.Client.Tests.ServerSentEventsTransportTests.SSETransportStopsSendAndReceiveLoopsWhenTransportStopped
Error Message:
 System.TimeoutException : Operation in SSETransportStopsSendAndReceiveLoopsWhenTransportStopped timed out at C:\projects\signalr\test\Microsoft.AspNetCore.SignalR.Client.Tests\ServerSentEventsTransportTests.cs:117
Stack Trace:
   at Microsoft.AspNetCore.SignalR.Tests.Common.TaskExtensions.<OrTimeout>d__2.MoveNext() in C:\projects\signalr\test\Common\TaskExtensions.cs:line 26
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
   at Microsoft.AspNetCore.SignalR.Client.Tests.ServerSentEventsTransportTests.

Yes - we need new corefx packages to fix that.


Add model auto redeploy feature (#852)

Description
Backport PR https://github.com/opensearch-project/ml-commons/pull/852 to main branch
Issues Resolved
Check List

[ ] New functionality includes testing.

[ ] All tests pass

[ ] New functionality has been documented.

[ ] New functionality has javadoc added

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

DCO is failing.


MM-49632 Don't block center channel during team switch

When switching teams, we reload all of the current users' channels and channel members for that team (including DMs/GMs). I think there's a good chance that this is no longer needed, but I'm not certain about that yet, so that'll be a future ticket, probably as part of the one OKR Zubair has for the upcoming quarter.
This recently started feeling slower on Community because GraphQL requires paging through those channels and making multiple requests which are blocked on each other. To mitigate this, I just changed it so that we don't show the "Loading..." when switching teams any more. We're not making any less requests, but it'll improve perceived performance.
Ticket Link
https://mattermost.atlassian.net/browse/MM-49632
Also lets us close https://mattermost.atlassian.net/browse/MM-48260
Release Note
Don't show center channel as loading while switching teams

E2E tests not automatically triggered, because PR has no approval yet. Please ask a developer to review and then try again to attach the QA label.

/e2e-tests

Successfully triggered E2E testing!
GitLab pipeline | Test dashboard

Nope, there's no more changes planned. I'll wait for Zubair to confirm, but I think the issue we ran into before was that guest users would see the centre channel stuck as "Loading..." because one of those requests failed and state.requests.channels.getChannelsAndChannelMembers would get set to something other than success. Since we don't check that state before for the loading indicator, that wouldn't happen any more regardless of if a request failed.
That said, I couldn't reproduce the issue locally, so perhaps it was already fixed some other way

/update-branch

The guest issue is not able to reproduce in this PR. so may be it got fixed.

Thanks @hmhealey
Verified team switching is working as expected.
I also tried to create few guest users to see if the issue mentioned above is fixed.
I used "move to" to sort guests channels, and see one error when guest is moving channel to new category (3-dot menu>Move to>new category). Guest user belonged to 2 teams, 1 channel on each team.
I couldn't repro on master.

E2E report - no PR related failures

Thanks @hmhealey
This PR is ok to merge then. I'll follow up with Zubair on menu issue.


Service call to get all parameters

Would be nice to have an option to retrieve all parameters at once (names, values, and types)

Feature currently exists.


CI: INIT_NEWCONTAINER failed

I0426 02:27:16.475067   22846 vm_states.go:47] SB[vm-pdLbyEynVG] sent INIT_NEWCONTAINER
E0426 02:27:16.475134   22846 container.go:252] Pod[busybox] Con[1416cf5306a1(busybox-busybox-aZRQwfjqHE)] failed to start container: Create new container failed: Error:
I0426 02:27:16.475175   22846 vm_console.go:96] SB[vm-pdLbyEynVG] [CNL] hyper_modify_event modify event fd 3, 0x61d648, event 8193
E0426 02:27:16.475233   22846 provision.go:482] Pod[busybox] error during start all containers: finished with errors: map[1416cf5306a146aaf4fd255e26d512dde0fe374ef2c92b7015902493407d3159:Create new container failed: Error: ]
I0426 02:27:16.475287   22846 vm_states.go:55] SB[vm-pdLbyEynVG] Con[1416cf5306a146aaf4fd255e26d512dde0fe374ef2c92b7015902493407d3159] container finished, unset iostream pipes
I0426 02:27:16.475325   22846 vm.go:206] SB[vm-pdLbyEynVG] got shut down msg, acked here
I0426 02:27:16.475379   22846 container.go:1066] Pod[busybox] Con[1416cf5306a1(busybox-busybox-aZRQwfjqHE)] container exited with code 255 (at 2017-04-26 02:27:16.475308592 +0000 UTC)
I0426 02:27:16.475404   22846 container.go:1072] Pod[busybox] Con[1416cf5306a1(busybox-busybox-aZRQwfjqHE)] clean up container
I0426 02:27:16.475428   22846 run.go:55] failed to  start pod busybox: finished with errors: map[1416cf5306a146aaf4fd255e26d512dde0fe374ef2c92b7015902493407d3159:Create new container failed: Error: ]
E0426 02:27:16.475440   22846 pod.go:34] StartPod failed: finished with errors: map[1416cf5306a146aaf4fd255e26d512dde0fe374ef2c92b7015902493407d3159:Create new container failed: Error: ]
hyper_test.go:517:
    c.Assert(err, IsNil)
... value *grpc.rpcError = &grpc.rpcError{code:0x2, desc:"finished with errors: map[1416cf5306a146aaf4fd255e26d512dde0fe374ef2c92b7015902493407d3159:Create new container failed: Error: ]"} ("rpc error: code = Unknown desc = finished with errors: map[1416cf5306a146aaf4fd255e26d512dde0fe374ef2c92b7015902493407d3159:Create new container failed: Error: ]")

I've seen it a few times and the latest one is here:
https://travis-ci.org/hyperhq/hyperstart/jobs/225846312

turns out it is a dup of https://github.com/hyperhq/hyperd/issues/601


Fix tracking of install

first event time <--> warmup
last event time <--> cooldown
I don't know if there's a way to enforce these relationships using the type system without making everything just a restrictive (i.e. 1-1) as before...


I want to see all microblogs they have written recently like twitter.Help.

I have followed some users.
I want to see all microblogs they have written recently like twitter.
I think it is importent and common.
Can you help me?

I resolve it myself.Thank you all the same。
class LikeController < ApplicationController
  layout 'like'

  def index
    following = Follow.where(["follower_id = ?", @user.id])
    following_ids = following.collect{|f| f.followable_id}
    @blogs = Blog.where(:user_id => following_ids).order('created_at DESC').limit(20)
  end
end


random freezes of phantomJS during tests in Docker Container

Hello!
We are running Behat tests in our CI system with the PhantomJS 2.1.1 binary (PHP/Behat/Mink/PhantomJS) on Docker containers. But very often, there are random freezes of PhantomJS. The process is still running, but doesn't respond anymore, so the Behat test is hanging. Our containers are running on ubuntu host machines on AWS EC2.
Initially we thought, that the error occured only when all the memory of the container was used, but later we found out that the problem also occurs with enough free memory.
We already tried to restart PhantomJS at every Behat test case, but this doesn't prevent the error either.
Is there perhaps some known issue about this? Don't hesitate to get back to me if you need more information.
Thank you,
Pascal

If you can get to your logs, do you see something like __cleanupWindowlessSessions - Asynchronous cleanup starting NOW

We're having issues with phantomjs in docker as well. Our tests will hang at an indeterminate point such as:
2016-10-06T18:57:58 [DEBUG] WebPage - updateLoadingProgress: 33
2016-10-06T18:57:58 [DEBUG] WebPage - updateLoadingProgress: 36
2016-10-06T18:57:58 [DEBUG] WebPage - updateLoadingProgress: 38
2016-10-06T18:57:58 [DEBUG] WebPage - updateLoadingProgress: 40
2016-10-06T18:57:58 [DEBUG] WebPage - updateLoadingProgress: 42

I'm not seeing anything about windowless sessions.
It'd be great to have a build off of master we could test against to see if this is fixed.

I have an open issue that started in much the same way. I ran phantom js from selenium separately and found that as an issue. https://github.com/ariya/phantomjs/issues/14588.

@asevans48 Indeed... I added some logging and found this:
[INFO  - 2016-10-19T20:29:06.182Z] SessionManagerReqHand - _cleanupWindowlessSessions - Asynchronous Sessions clean-up phase starting NOW

@asevans48 It seems to me that this is the same problem than the one mentionned in #11526

Because it is happening in more than just Behat, my system is straight java with no containers yet. For me it seems to be a concurrency problem. I am going to try LXC.

I've also experienced random crashes with Docker; we've been able to reproduce locally on OSX hosts, though local crashes are not as common as those running on EC2 Ubuntu hosts


Exception in thread "main" brut.androlib.AndrolibException: Could not decode arsc file

Hamids-MBP:apk Hamid$ ./apktool d roqya.apk
I: Using Apktool 2.0.0 on roqya.apk
I: Loading resource table...
Exception in thread "main" brut.androlib.AndrolibException: Could not decode arsc file
	at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:52)
	at brut.androlib.res.AndrolibResources.getResPackagesFromApk(AndrolibResources.java:538)
	at brut.androlib.res.AndrolibResources.loadMainPkg(AndrolibResources.java:63)
	at brut.androlib.res.AndrolibResources.getResTable(AndrolibResources.java:55)
	at brut.androlib.Androlib.getResTable(Androlib.java:64)
	at brut.androlib.ApkDecoder.setTargetSdkVersion(ApkDecoder.java:209)
	at brut.androlib.ApkDecoder.decode(ApkDecoder.java:92)
	at brut.apktool.Main.cmdDecode(Main.java:165)
	at brut.apktool.Main.main(Main.java:81)
Caused by: java.io.IOException: Expected: 0x00000008, got: 0x00000001
	at brut.util.ExtDataInput.skipCheckShort(ExtDataInput.java:56)
	at brut.androlib.res.decoder.ARSCDecoder.readValue(ARSCDecoder.java:238)
	at brut.androlib.res.decoder.ARSCDecoder.readEntry(ARSCDecoder.java:201)
	at brut.androlib.res.decoder.ARSCDecoder.readConfig(ARSCDecoder.java:189)
	at brut.androlib.res.decoder.ARSCDecoder.readType(ARSCDecoder.java:157)
	at brut.androlib.res.decoder.ARSCDecoder.readPackage(ARSCDecoder.java:114)
	at brut.androlib.res.decoder.ARSCDecoder.readTable(ARSCDecoder.java:78)
	at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:47)
	... 8 more

Looks like duplicate of #1121 , but I haven't got an APK for either.

I'm getting the same issue on the new TW M builds with the latest apktool release and master branch:

I: Using Apktool 2.1.0-a64a03-SNAPSHOT on framework-res.apk
I: Loading resource table...
Exception in thread "main" brut.androlib.AndrolibException: Could not decode arsc file
at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:52)
at brut.androlib.res.AndrolibResources.getResPackagesFromApk(AndrolibResources.java:544)
at brut.androlib.res.AndrolibResources.loadMainPkg(AndrolibResources.java:63)
at brut.androlib.res.AndrolibResources.getResTable(AndrolibResources.java:55)
at brut.androlib.Androlib.getResTable(Androlib.java:67)
at brut.androlib.ApkDecoder.setTargetSdkVersion(ApkDecoder.java:198)
at brut.androlib.ApkDecoder.decode(ApkDecoder.java:96)
at brut.apktool.Main.cmdDecode(Main.java:165)
at brut.apktool.Main.main(Main.java:81)
Caused by: java.io.IOException: Expected: 0x00000008, got: 0x00000000
at brut.util.ExtDataInput.skipCheckShort(ExtDataInput.java:56)
at brut.androlib.res.decoder.ARSCDecoder.readValue(ARSCDecoder.java:282)
at brut.androlib.res.decoder.ARSCDecoder.readEntry(ARSCDecoder.java:224)
at brut.androlib.res.decoder.ARSCDecoder.readTableType(ARSCDecoder.java:212)
at brut.androlib.res.decoder.ARSCDecoder.readTableTypeSpec(ARSCDecoder.java:154)
at brut.androlib.res.decoder.ARSCDecoder.readTablePackage(ARSCDecoder.java:116)
at brut.androlib.res.decoder.ARSCDecoder.readTableHeader(ARSCDecoder.java:78)
at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:47)
... 8 more

Apk: https://www.dropbox.com/s/i80xi2uyjlfttji/framework-res.apk?dl=0

@wanam
Use Apktool v2.0 for TW MM framework-res & other file

@mythtrandyr Nice catch, 2.0.0 is working fine.
@iBotPeaches Is this a regression?

Confirmed.

Yeah this is strange and regression. I thought the problem occurred during my massive changes in 2.0.3, but this is isolated to changes between 2.0.0 and 2.0.1. More importantly during the read of ConfigFlags.
Working on a fix.

I have a similar issue. Would an additional APK sample help?

@iBotPeaches Having the same issue with the Galaxy S7. I can install and decompile frameworks and secsettings with 2.0.0, other apks get the same error as above. If I switch to 2.0.3 or build form source, after installing frameworks, I receive a non stop entire apk of the following error (SecContacts was used in screenshot. Terminal wouldnt let me copy)
screenshot- https://drive.google.com/file/d/0B3FpYTXXDE_ARTlVYWtsM2o2R3M/view?usp=sharing
apks- https://drive.google.com/file/d/0B3FpYTXXDE_AbjF0NEpqNEQ2a1E/view?usp=sharing

This broke because of this commit - https://github.com/iBotPeaches/Apktool/commit/6ee029dd30808a05f842c46672261a27dbb107da
If we revert this commit we break #924
On this bug.
76 bytes read (reverted) = good.
72 bytes read (not reverted) = bad
On #924
72 bytes read (reverted) = good
68 bytes read (not reverted) = bad
So patch or not something gets broken. I need to figure out why this is, but I now know the problem. Sorry for the delay.

Any update on this @iBotPeaches ?

Unfortunately nothing easy. Like I had expected, simply reverting the commit that caused the problem this far down the commit line doesn't work. Trying to make this work with the current codebase I don't think is the best method of attack as tons of refactoring has occurred around it.
We just have big problem in which some APKs rely on reading the remaining bytes in a ResConfig while others rely on skipping the reading of these extra bytes. This is a combination of KNOWN_BYTESand this fix. I'm thinking of a command line parameter for advanced users that sets KNOWN_BYTES so if these edge cases present themselves they can be solved with some googling and a different parameter.

Okay spent a good deal of time this weekend on this. While I don't have a fix, I have good news. I've isolated the problem down to this struct - https://github.com/android/platform_frameworks_base/blob/master/include/androidfw/ResourceTypes.h#L897
Before that function runs we are position 20 (byte) of the reader, at the end of it. We are at 72. 72 - 20 = 52 bytes.
    private static final int KNOWN_CONFIG_BYTES = 52;

That is exactly whats expected, so the problem doesn't make much sense. Before this function runs we have - https://github.com/iBotPeaches/Apktool/blob/6ee029dd30808a05f842c46672261a27dbb107da/brut.apktool/apktool-lib/src/main/java/brut/androlib/res/decoder/ARSCDecoder.java#L171
Or entriesStart variable. This allows us to skip the entire block if needed since we know where the next "block" begins. For this instance we see the entriesStart at 5140 (0x1414), but our actual value is 5136 or (0x1410). As you can see, there is the 4 missing bytes.
The entire time I spent looking for the missing 4 bytes I failed. Skipping these bytes might prove to cause a problem, but we know the mistake happens at this struct - https://github.com/android/platform_frameworks_base/blob/master/include/androidfw/ResourceTypes.h#L897
You can compare that struct to our reading here - https://github.com/iBotPeaches/Apktool/blob/6ee029dd30808a05f842c46672261a27dbb107da/brut.apktool/apktool-lib/src/main/java/brut/androlib/res/decoder/ARSCDecoder.java#L266
After multiple multiple checks, there is no error there.
So the fix. Currently our DataInput has no counter. I need to add a decorator or wrap it in a counting stream so I can get current position. If after key points in apktool's execution if the next position doesn't match whats expected we either have a bug, problem or a place to simply skip bytes and hope for the best.
Couple notes

No I can't just raise KNOWN_CONFIG_BYTES. That matches the size variable of the ResConfig which is still 52.
Yes the 52 is right. The read starts at position 20 and ends at 72. 72 - 20 = 52.
Does the size (4 bytes) count towards that? As of now yes, but that would explain the missing 4 bytes. However simply skipping those 4 bytes break apks everywhere.

➜  Bug1131 apktool d framework-res.apk -f
I: Using Apktool 2.1.1-032a3e-SNAPSHOT on framework-res.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...
➜  Bug1131

Due to the complexity of this change. I'll need to test this a lot more, but it is working for the test suite and the original apk here. So thats good news.

➜  G930F_Galaxy_S7 apktool if framework-res.apk -t 1131
I: Framework installed to: /home/ibotpeaches/apktool/framework/1-1131.apk
➜  G930F_Galaxy_S7 apktool if twframework-res.apk -t 1131
I: Framework installed to: /home/ibotpeaches/apktool/framework/2-1131.apk
➜  G930F_Galaxy_S7 apktool d SecContacts_M.apk -t 1131
I: Using Apktool 2.1.1-032a3e-SNAPSHOT on SecContacts_M.apk
I: Loading resource table...
I: Decoding Shared Library (touchwiz), pkgId: 2
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: /home/ibotpeaches/apktool/framework/1-1131.apk
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
I: Baksmaling classes.dex...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...
➜  G930F_Galaxy_S7 apktool d SystemUI.apk -t 1131
I: Using Apktool 2.1.1-032a3e-SNAPSHOT on SystemUI.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: /home/ibotpeaches/apktool/framework/1-1131.apk
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
I: Baksmaling classes.dex...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...
➜  G930F_Galaxy_S7

The other application in this thread. Proving both work with this patch. I'll clean it up and merge it in over the weekend / tonight.

Sorry to be a pain and open an old bug report, I'm having the same issue with the Sony Select APK.
~/bin/apktool d ~/Downloads/apk/Sony\ Select_5.0.7.zip --output ~/Downloads/apk/SonySelect.out
I: Using Apktool 2.2.1 on Sony Select_5.0.7.zip
I: Decoding AndroidManifest.xml with only framework resources...
Exception in thread "main" brut.androlib.AndrolibException: Could not decode XML
	at brut.androlib.res.decoder.XmlPullStreamDecoder.decode(XmlPullStreamDecoder.java:147)
	at brut.androlib.res.decoder.XmlPullStreamDecoder.decodeManifest(XmlPullStreamDecoder.java:153)
	at brut.androlib.res.decoder.ResFileDecoder.decodeManifest(ResFileDecoder.java:140)
	at brut.androlib.res.AndrolibResources.decodeManifest(AndrolibResources.java:162)
	at brut.androlib.Androlib.decodeManifestFull(Androlib.java:117)
	at brut.androlib.ApkDecoder.decode(ApkDecoder.java:120)
	at brut.apktool.Main.cmdDecode(Main.java:166)
	at brut.apktool.Main.main(Main.java:81)
Caused by: java.io.IOException: Expected: 0x00080003, got: 0x6d783f3c
	at brut.util.ExtDataInput.skipCheckInt(ExtDataInput.java:48)
	at brut.androlib.res.decoder.AXmlResourceParser.doNext(AXmlResourceParser.java:843)
	at brut.androlib.res.decoder.AXmlResourceParser.next(AXmlResourceParser.java:100)
	at brut.androlib.res.decoder.AXmlResourceParser.nextToken(AXmlResourceParser.java:110)
	at org.xmlpull.v1.wrapper.classic.XmlPullParserDelegate.nextToken(XmlPullParserDelegate.java:105)
	at brut.androlib.res.decoder.XmlPullStreamDecoder.decode(XmlPullStreamDecoder.java:140)
	... 7 more

➜  Bug1131 apktool d Sony\ Select_5.0.7.apk
I: Using Apktool 2.2.2-7c6c7a-SNAPSHOT on Sony Select_5.0.7.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: /home/ibotpeaches/.local/share/apktool/framework/1.apk
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...
➜  Bug1131 apktool d Sony\ Select_5.0.7.apk --output foo
I: Using Apktool 2.2.2-7c6c7a-SNAPSHOT on Sony Select_5.0.7.apk
I: Loading resource table...
I: Decoding AndroidManifest.xml with resources...
I: Loading resource table from file: /home/ibotpeaches/.local/share/apktool/framework/1.apk
I: Regular manifest package...
I: Decoding file-resources...
I: Decoding values */* XMLs...
I: Copying assets and libs...
I: Copying unknown files...
I: Copying original files...
➜  Bug1131

Not sure @hoshsadiq , couldn't duplicate with --output or not.

Very odd hm. I'll try and replicate it again and tie down the causes if you'd like! Will report back. I think it may be because I was giving it a path as a value for --output where --output actually just requires a dir name (correct me if I'm wrong).

I have a similar issue. After i deleted resources.arsc, it passed.

Since 2.5.1 has not released, I worked-around with this tutorial. Hope it helps someone.
https://platinmods.com/threads/how-to-fix-apktool-decompile-error-using-mt-manager-app-arscdecoder-error.121708/

Still not fixed in 2.7.1

@iBotPeaches getting the following error in the version 2.9.0. is this issue related to it?
I: Using Apktool 2.9.0 on 125358564fdedf675613b99233c8e8d8214f6187f94c18a733d040b80d39ebfb.apk
I: Loading resource table...
Exception in thread "main" brut.androlib.exceptions.AndrolibException: Could not decode arsc file
	at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:49)
	at brut.androlib.res.data.ResTable.loadResPackagesFromApk(ResTable.java:182)
	at brut.androlib.res.data.ResTable.loadMainPkg(ResTable.java:134)
	at brut.androlib.res.ResourcesDecoder.decodeResources(ResourcesDecoder.java:147)
	at brut.androlib.ApkDecoder.decode(ApkDecoder.java:104)
	at brut.apktool.Main.cmdDecode(Main.java:217)
	at brut.apktool.Main.main(Main.java:92)
Caused by: java.util.zip.ZipException: ZipFile invalid LOC header (bad signature)
	at java.base/java.util.zip.ZipFile$ZipFileInputStream.initDataOffset(ZipFile.java:935)
	at java.base/java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:946)
	at java.base/java.io.BufferedInputStream.fill(BufferedInputStream.java:291)
	at java.base/java.io.BufferedInputStream.implRead(BufferedInputStream.java:325)
	at java.base/java.io.BufferedInputStream.read(BufferedInputStream.java:312)
	at java.base/java.io.FilterInputStream.read(FilterInputStream.java:71)
	at org.apache.commons.io.input.ProxyInputStream.read(ProxyInputStream.java:165)
	at com.google.common.io.LittleEndianDataInputStream.readAndCheckByte(LittleEndianDataInputStream.java:232)
	at com.google.common.io.LittleEndianDataInputStream.readUnsignedShort(LittleEndianDataInputStream.java:101)
	at com.google.common.io.LittleEndianDataInputStream.readShort(LittleEndianDataInputStream.java:194)
	at brut.util.DataInputDelegate.readShort(DataInputDelegate.java:46)
	at brut.androlib.res.data.arsc.ARSCHeader.read(ARSCHeader.java:46)
	at brut.androlib.res.decoder.ARSCDecoder.nextChunk(ARSCDecoder.java:668)
	at brut.androlib.res.decoder.ARSCDecoder.readResourceTable(ARSCDecoder.java:72)
	at brut.androlib.res.decoder.ARSCDecoder.decode(ARSCDecoder.java:44)

APK file : https://www.dropbox.com/scl/fo/5wymbvrd8nnpz0jrl74yz/h?rlkey=wzz6puinj0bwfukduuuktbcao&dl=0

@harsh-im - It is not. You have an archive (zip) that is being rejected by the increasing validation that the Zip Class offers in Java. Now AOSP/Android might accept it, but stricter Java validation may not.
What you are encountering is basically this ticket - https://github.com/iBotPeaches/Apktool/issues/2553, but there is no good workaround at the moment as only option is to build a Zip class that doesn't invoke all the same criteria during parsing.

Thank you for getting back to me quickly, @iBotPeaches. You mentioned that the error is caused by stricter validation in the Zip class in some newer Java versions. Based on the discussion in ticket #2553, can I resolve this issue by switching to a different Java version when using apktool?

Thank you for getting back to me quickly, @iBotPeaches. You mentioned that the error is caused by stricter validation in the Zip class in some newer Java versions. Based on the discussion in ticket #2553, can I resolve this issue by switching to a different Java version when using apktool?

Maybe?
The LTS versions were patched as well.