Upgrade Mockito 5.14.0 -> 5.14.2

This PR contains the following updates:

Package
Type
Update
Change

Mockito

patch
5.14.0 -> 5.14.2

Mockito
import
patch
5.14.0 -> 5.14.2

Release Notes

mockito/mockito (Mockito)
v5.14.2
Compare Source
Changelog generated by Shipkit Changelog Gradle Plugin
5.14.2

2024-10-15 - 12 commit(s) by Brice Dutheil, Rafael Winterhalter, dependabot[bot]
Fix #3466 nexus publishing configuration (#3470)
Bump org.jetbrains.kotlin:kotlin-stdlib from 2.0.20 to 2.0.21 (#3468)
Bump bytebuddy from 1.15.3 to 1.15.4 (#3467)
Missing 5.14.2 release (#3466)
chore: Tests whether JVM warnings / messages on dynamic attach (#3462)
Bump junit-jupiter from 5.11.1 to 5.11.2 (#3461)
Renames extension modules with mockito- prefix (#3460)
Avoid attach warning if Byte Buddy is configured for command-line attach. (#3459)
Bump org.shipkit:shipkit-auto-version from 2.0.10 to 2.0.11 (#3458)
Bump junit-jupiter from 5.11.0 to 5.11.1 (#3455)
Move root project to dedicated core folder (#3444)
Bump biz.aQute.bnd:biz.aQute.bnd.gradle from 6.4.0 to 7.0.0 (#3136)

v5.14.1
Compare Source
Changelog generated by Shipkit Changelog Gradle Plugin
5.14.1

2024-09-30 - 2 commit(s) by Brice Dutheil, dependabot[bot]
fix: gradle mockitoAgent configuration should not be transitive (#3454)
Bump bytebuddy from 1.15.2 to 1.15.3 (#3452)
Allow for installing a Java agent within the Mockito jar, without exposing Byte Buddy's attach mechanism. (#3437)

[ ] If you want to rebase/retry this PR, check this box

Suggested commit message:
Upgrade Mockito 5.14.0 -> 5.14.2

See:
- https://github.com/mockito/mockito/releases/tag/v5.14.1
- https://github.com/mockito/mockito/releases/tag/v5.14.2
- https://github.com/mockito/mockito/compare/v5.14.0...v5.14.2


add 'manage topic' link to errored topic screen

There is currently no way to manage an errored topic.

where exactly hal? We do show errors on the all topics landing page and you can click into summary page..

It somehow happens to me pretty often that I create a topic and end up
wanting to manage it even through it has not finished spidering yet.
usually this is because there is some sort of error with it, but sometimes
it is just because it is spidering.  I should be able to get to the
management screen any time, I think.
-hall
On Thu, Mar 8, 2018 at 2:14 PM, rahulbot<EMAIL_ADDRESS>wrote:

This is the idea for #892
https://github.com/mitmedialab/MediaCloud-Web-Tools/issues/892 - that
when a topic is in error state, we show a note and big button to drop them
back into the topic creation wizard, but this time they are editing the
topic they created (but didn't work).
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/mitmedialab/MediaCloud-Web-Tools/issues/912#issuecomment-371592694,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABvvTy5xpRlV0sjKLTRuZKW9LgsCBbFhks5tcYMvgaJpZM4SjH59
.

--
Hal Roberts
Fellow
Berkman Klein Center for Internet & Society
Harvard University

this is pretty complicated b/c our TopicContainer brokers the topic.state, and is currently not allowing any state but "completed" pass through to the EditTopicContainer
"created but not queued" and "queued" could allow editing, but not "running". However, if the user then either intentionally or unintentionally constrains the query, we have to then offer a reset - which is difficult to infer since the state is not officially in "error"


A few updates

Hi Rosey,
I sent a few updates via email if you would like to incorporate, and just so you know its not spam, I am also writing here

Added support for Images
Added support for UNDERLINE, SUBSCRIPT, SUPERSCRIPT
Added support for custom inline styles (currently color, background color) - rendered as <span elements for markdown
Enabled html tag remarkable handling (for the above need to embed html tags such as span)
Allowing for custom html block and inline style entities for end-users
Fixed embedding styles for example ***BOLD AND ITALIC - previously the markdown-to-draft was not adjusting correctly the length of the inline style

https://www.dropbox.com/s/65t2k8c1lpmfn6h/Archive.zip

Added support for UNDERLINE, SUBSCRIPT, SUPERSCRIPT
Just going to close this as there's a pull request with these changes and further discussion

I don't see any reference to UNDERLINE in the code...


Feedback: Funding Agency Name and Code

This is the place to leave your feedback and questions about the following data elements, feedback will close on August 11, 2015:

Funding Agency Name
Funding Agency Code

The proposed definition of the above elements:
http://fedspendingtransparency.github.io/whitepapers/fundingagency/
Federal Spending Collaboration home page: http://fedspendingtransparency.github.io/

During Recovery Act implementation, OMB and the Recovery Board addressed the question of "Funding Agency" vs "Awarding Agency".  To provide best clarity for the recipients and the public, we applied these as:
Funding Agency is the agency of the Treasury Account to which the funds are appropriated by Congress.  This agency may circuit the funds to other agencies for implementation.
Awarding Agency is the agency that interacts with the recipient to make the award and manage the assistance (grants, etc.).  The recipient reports to the Awarding Agency.
We found this to best illustrate the movement of taxpayers' funds from Congress to project/recipient, while being comprehensible to the recipients submitting reports and facilitating the public review and analysis of data.  In most cases, the funding agency and the awarding agency are the same entity.

Funding Agency Code (or Name) should reflect a high degree of consistency across all report audiences -- either budget and accounting, grant oriented, or acquisition focused.  We certainly can't expect the general public to get acquainted with all the wrinkles that present government-wide coding allows and the unique implementations each of our agencies contributes.  So I thought I would follow up on the previous commenter's thread and check for commonalities (or not) between coding systems in budget versus contracts.  There are some notable disconnects between authoritative agency codes in budget arena (reference OMB Circular No. A-11) and what I found over in the contract reporting area (FPDS-NG hosted by GSA).  Agency codes in FPDS are apparently 4 digit wide and primarily numeric but can be alpha-numeric.  Referring to Appendix C of A-11, we find OMB agency codes are 3 digit wide all numeric domain (related Bureau codes are 2 digit wide).  Associated Treasury agency codes are 2 digits wide and the upcoming CGAS agency code expands the Treasury code to 3 digits.  The OMB agency code for Department of Agriculture is '005', the Treasury code is '12', the CGAS code variant is '012' and the FPDS contracting/funding agency code is '12C2'.  (One positive omen is that the first two characters of the FPDS funding agency code '12' mirror the values in the traditional Treasury agency code but the following two digits showing 'C2' disconnect from OMB Bureau code for Forest Service of '96').
As one of the first steps to synchronizing data element across agencies and their enterprise systems, it would make sense to agree on one field definition and defer to one of the government-wide "authoritative" actors.  For example, I can see favoring a 3-digit CGAS version of Treasury code over OMB code insofar as "spending" relates more to budget execution than formulation and the latter is where OMB codes seem to correlate to OMB MAX Data Entry application. We need to reconcile "agency [funding] code" between Treasury, OMB and GSA if there is to any hope of standardizing Federal agency systems' data element width, type, and valid range.  The column heading "agency code" should not require contextual interpretation according to who sent the data to the repository.  A snippet of an exported standard report issued from the FPDS web site at: https://www.fpds.gov/fpdsng_cms/index.php/en/ is provided below.  Also furnished is an image snippet below from Appendix C of the OMB Circular No. A-11 as revised 2015 found at: https://www.whitehouse.gov/sites/default/files/omb/assets/a11_current_year/2015_letter.pdf .

Observations shared regarding Funding Agency Code in my previous post also apply to Funding Agency Name.  We can start with the most mundane reconciliation concerns such as requiring standard content formatting.  The first-order of business will be to cross-check names listed for agencies by OMB, Department of Treasury, and GSA in their system dictionaries.  As noted before -- prior to seeking consensus from the "feeder: agencies, the various "authorities" need to come together.  Is "Department of Treasury", "Dept. of Treasury", and "Treasury Department" equally valid?  Can they be post-processed into a fixed name by the repository import module as opposed to requiring commonality in the source data?
Should each transmitting organization have their enterprise system(s) store name as entirely capitalized or in Proper case?  Then a fine-point as to field display treatment--does the future massive repository's reporting tool post-process the uploaded value and translate to all caps or in Proper case?  Do we impose a fixed length string data type, e.g. CHAR (50) or allow some flexibility and use VARCHAR(254) or get loose with TEXT type for this data element?  And expansion of abbreviations (or not) from the transmit file is a potential trip point.  Should we require full spell out of a name or allow -- or -- enforce limited abbreviation?  One system could retain values such as 'Dept.' and our hypothetical business rules would detect the item and decode the phrase to 'Department'.  What about trailing blanks lurking on the upload?  The repository could apply TRIM() functions on upload data before storage (or not).  This all sounds very "IT" but achieving consistent names will be even more challenging than arriving at consensus for code.

I am wondering why the definition for Awarding Agency and the one for FUnding Ageny are not 'parallel'.  Funding Aengy describes a characteristic (provides most funding to an award) whereas Awarding AGency describes source of name/code (from the treasury symbol)   I assume that the Funding agency name/code is based on the treasury symbol too.  So how is Awarding Agency defined in comparison to FUnding Agency?  Does Awarding agency give funds to the award but not as much as the Funding?
Do we get answers here? or do answers go to our email address?
KM

AGA established a work group charged with responding to inquiries by the federal government, regarding implementation of the DATA Act. The following response is being submitted on behalf of AGA's work group, which is comprised of state, local, tribal and university practitioners from across the country. In commenting on this topic, group members had the following comments:
•	Multiple uses of the term “agency” are confusing. The definition of “Federal Agency” in the Act itself refers to “Executive Agency.” In addition, the elements include “Funding Agency” and “Awarding Agency.”

@MeadowK I don't have an answer to your question, but wanted to let you know that all conversation will happen here on this page, rather than through e-mail (so everyone can take part).

I want to offer a big +1 to @MJB-NIH 's comments.  Hopefully the government is providing an authoritative source for reporting attributes, like names, that agencies can use when they report data.  This master data is important for consistency and managed by the data governance body.
The data within FPDS has multiple names for the same agency for example.  This propagates to USASpending.  Basic data management principles and master data should be applied before data is published to the public to minimize these data quality issues (if they aren't applied to the data as it is input to the systems).  Ideally, a lead agency will assume responsibility for managing this data and provide it as a service to other agencies for use in their consistent reporting.

are we addressing Reimbursable agreements. Dept. of Justice has shared grants with other federal agencies so who is that responsible reporting entity.

+1 to @MJB-NIH comments regarding Funding Agency Name.  Dept of Interior shares concerns regarding master data management.

Update 8/11/2015: Thank you for your feedback on the data element Funding Agency Name and Code. We will consider your feedback as we continually assess how we can improve data definitions.
You may still contribute the discussion for other data elements. Check http://fedspendingtransparency.github.io/dataelements/ to find data elements with open feedback pages.


PHP7 code generation?

Hi there
A massive thank you for your work on the library to date - I came from using the DrSlump lib from a few years back which isn't maintained much, so it's been a great help to have a modern alternative!
I wanted to see what you thought about generating php7 code - is it in the near future or would you accept a pull request for this?

Hi..
That is not on my radar for the near future.
But of course I would review/accept a PR.
Also notice that google added support php..
Not sure how stable it is at this point and how well it support php7 but it might be another option if using a extension is ok https://github.com/google/protobuf/tree/master/php

I have considered that library, but some of the projects I maintain are v2 format which doesn't seem to be supported by google.


UTC Time

Is there any way to bypass timezone detection, and take input as UTC time?

Closing all issues. The plugin was a proof of concept which others have forked and made better alternatives.


Fix some issues reported by lintian

Fix some issues reported by lintian

Trim trailing whitespace. (file-contains-trailing-whitespace)
Use secure copyright file specification URI. (insecure-copyright-format-uri)
Bump debhelper from old 11 to 12. (package-uses-old-debhelper-compat-version)
Set debhelper-compat version in Build-Depends. (uses-debhelper-compat-file)
Set upstream metadata fields: Bug-Database, Bug-Submit, Repository, Repository-Browse. (upstream-metadata-file-is-missing)
Update standards version to 4.4.1, no changes needed. (out-of-date-standards-version)
Update standards version to 4.5.0, no changes needed. (out-of-date-standards-version)

This merge proposal was created automatically by the Janitor bot.
You can follow up to this merge proposal as you normally would.
Build and test logs for this branch can be found at
https://janitor.debian.net/lintian-fixes/pkg/kthresher/a0d734e9-c4fb-4ab8-968a-24d97b71f120.
These changes have no impact on the binary debdiff.
You can also view the diffoscope diff (unfiltered).

This merge proposal will be closed, since all remaining changes have been
applied independently.


None of the commands that uses pre-built packages runs on FreeBSD

🐛 Bug Report
The pre-built packages does not exist for FreeBSD and there is no clear way to build them yourself.
Furthermore they seem to segfaults after failing.
Environment

operating system: FreeBSD 12.1-RELEASE-p3 GENERIC  amd64
output of rustc -V: rustc 1.44.1 (c7087fe00 2020-06-17)
output of node -v: Not installed
output of wrangler -V: wrangler 1.10.1
contents of wrangler.toml: Empty or populated with the following after using cargo-generate directly:

name = "rust"
type = "rust"

account_id = ""
workers_dev = true
route = ""
zone_id = ""

Steps to reproduce
Install wrangler: cargo install wrangler
Attempt: wrangler generate
$ wrangler generate my-app https://github.com/cloudflare/rustwasm-worker-template
  Installing cargo-generate v0.5.0...
Error: could not download `cargo-generate`
no prebuilt cargo-generate binaries are available for this platform
Segmentation fault (core dumped)

Attempt: wrangler preview
$ wrangler preview
  Installing wasm-pack v0.9.1...
Error: could not download `wasm-pack`
no prebuilt wasm-pack binaries are available for this platform

What did you expect to see?
I expected to see a way that made it possible to build the needed binaries myself.
What did you see instead?
An error that did not give a lot more information other than the operating system is not supported.
And a segfault.

Thanks for the report! One workaround for now is to install wasm-pack and cargo-generate on your own. If you're up to the latest version, Wrangler should just use your system version instead of downloading a new one 😄

That was the first I attempted, but wrangler does not appear to look for the tools outside of the wrangler cache located in ~/.cache/.wrangler/.
(12:06:52 </usr/home/erk/dev>) 0 $ cargo generate --version
cargo-generate 0.5.0
(12:07:15 </usr/home/erk/dev>) 0 $ wrangler generate
  Installing cargo-generate v0.5.0...
Error: could not download `cargo-generate`
no prebuilt cargo-generate binaries are available for this platform
Segmentation fault (core dumped)

I got it working by making a hard link with ln the binary for cargo-generate into ~/.cache/.wrangler/cargo-generate-0.5.0/cargo-generate should be noted that a symbolic link did not work and would also error out. Similarily I made a hard link for wasm-pack.
Note to other users wanting to get this running on FreeBSD:
Do not use rustup to install your toolchain. As FreeBSD is a tier 2 platform rust-lld is not build in CI. This means that wasm is not supported when installed that way. Either install the rust package with pkg or build it from ports with wasm support enabled to get it working. Also wasm-pack does not seem to search PATH for wasm-opt so disable it until that is fixed.

This issue should probably not be marked as stale as the issue have not been fixed as far as I know.

Hello all,
I'm seeing an identical issue to OP however my platform is an aarch64-unknown-linux-gnu host.
Workaround of symlinking the installed .cargo/bin/cargo-generate binary in to the .cache/.wrangler/cargo-generate-0.5.0/cargo-generate is sufficient to get wrangler generate to work.
My best guess is that something relating to get_installation function (maybe the get_wrangler_cache part?) in https://github.com/cloudflare/wrangler/blob/master/src/install/mod.rs isn't working. Because previous to the workaround being applied, the code was getting all the way to attempting to download a prebuilt binary for cargo-generate (despite me already having installed cargo-generate v0.5.0)
I also tried testing with the latest version of cargo-generate installed, and that resulted in the same exact error:
wrangler --verbose generate --type=rust version
⬇   Installing cargo-generate v0.5.0...
Error: could not download `cargo-generate`
no prebuilt cargo-generate binaries are available for this platform

So regardless of cargo-generate version installed, it was unable to find it, and defaulted to attempting to install a prebuilt binary for a platform with no prebuilt binaries advertised.

Closing as there is a newer version of Wrangler written in node.js, which should work across all architectures that support node.js.


Allow multiple spells to have the same priority

This is an odd one.  Playing in the wrath beta as my paladin. The issue I'm running into is there no way to create a rotation that allows me to easily switch a judgement spell off my bar one of the three judgement abilities to be on my bars.
Basically paladins get three different judgements (Justice, Light and Wisdom) and you might switch up what judgement your doing periodically.  They all share the same cooldown, and act as if you judged your current seal, the only difference is the effect on the mob you hit.  So what I'd like is the ability for RM to just highlight any of my judgement abilities, instead of just one, if that makes sense.

This is ideally what the "Cooldowns" are for.
They are not necessarily part of the rotation (though you could add an 'in combat' condition) - and they will highlight when necessary regardless of what's going on with the rest of your rotation.  And any or all cooldowns could be highlighted at once.

Yeah, problem is judgement is part of the rotation, so using a cooldown doesn't quiet fit with that. it has a priority above one thing or another, but just the actual spell you have to cast can change based on comp.

I am implementing this slightly different than what you are suggesting.
I am adding a 'No Action' option to the rotation.  SO, what you could do, is setup cooldowns for Justice, Light & Wisdom (highlighting only if none of them are up)
And then in your rotation, add a 'No Action' if none of the judgements are up.  This could also be achieved currently by adding a spell or item you don't have on your bars too.  But the change will make this more explicit.
This way, your rotation would essentially stop until you have a judgement up, and the cooldowns would be inactive if any of the judgements are up.
This is a lot simpler to implement and understand with how Rotation Master is written right now.  It is written so that one cooldown or one rotation step only highlights one thing (it could highlight multiple buttons if you for some reason have multiple copies of the same spell on your bars - but each step only highlights one spell).
It would be a big change to make steps highlight multiple spells and would make the addon more confusing to use.

Should be able to achieve this using the mechanism I mentioned above in 1.4.0 beta1


custom scrollview with gesture handler and reanimated

We're looking to make a vertical swipable scrollview with gesture handler and reanimated for fullscreen list items (ala Tiktok type navigation). Has anybody done this before? And would it be wise to attempt this with gesture handler and reanimated or is there a better approach? We were thinking of taking Interactable.js and using it as follows:

use the snappoints for as the start point for each item
use snappoint index for the list index
use velocity to detect fling gestures and permit users to swipe multiple items at a time

I don't know if building ScrollView from scratch is a good idea. This components are very platform-specific and has very subtle config which might be hard to reproduce. Also it might change with versions of iOS and Android.
I tried to make it as well and didn't manage to provide anything which might imitate UX of native ScrollView.
However, if you manage to to it, it would be extremely great and help in various ways (e.g. I wanted to make custom bottom sheet component and build own scrollView was one of scenarios for achieving it).

@osdnk thanks! I'll close this for now. If we manage to work this out, we'll share the code via a pull request.

https://github.com/osdnk/scroll-imitate
Feel free to help me to do it better :)

@osdnk That's wicked. I'll play around with it later today and see if what I can do.

The main thing is to make this vals https://github.com/osdnk/react-native-scroll-view-imitate/blob/master/App.js#L21 close to native.

Thanks @osdnk! We've taken a look at the code. We have a few suggestions to make with the velocity but it seems to be working quite well.
What we're wondering is how would you would go about implementing snap points with a flatlist for instance?
You have this here:
  snapTo = index => {
    if (!this.props.enabledManualSnapping) {
      return;
    }
    this.manuallySetValue.setValue(this.state.snapPoints[this.state.propsToNewIncides[index]]);
    this.isManuallySetValue.setValue(1);
  };

Would we pass the render items from a flatlist into this? Or, would we pass this into flatlist as a replacement scrollview, then pass this index as props into Flatlist and go from there?

It's doable. A made it because I was preparing for releasing https://github.com/osdnk/reanimated-bottom-sheet and I was my main idea. But I'll consider it. Also, I'm thinking about releasing reanimated-scroll-view for more complex customizations as you mentioned. But now I do not have a lot of time so maybe you'd like to help?

Sure, what would you like us to take a stab at

We're basically looking to achieve what you see in the video:
https://www.dropbox.com/s/hw0l1bv85r6v79g/swipe-scroll.mov?dl=0
Regular RN scrollview can handle doing this on iOS but not at all on Android. Hence, why we're attempting to use

@pribeh  Doable. Instead of snapping to index you might snap to point.
I think you might try this:
https://github.com/osdnk/react-native-scroll-view-imitate/blob/feadab303ba05c875ddc4f1b4685ae1a55124c3b/App.js from this commit. It's the last commit when the name of the repo was matching with content.
and customize in a way you want. If you'd like, send it later and I'll probably release it soon as a lib :)

@osdnk thanks. I'll try to see what I can do.

@osdnk Could you share a quick video of the bottom sheet you made. I could probably help with that.
I didn't have time to try your code but I think it's doable using Reanimated and Interactable without any scrollview and It will feel so smooth on any iOS or Android

Same goes for you @pribeh with TickTok Vertical Swiper. I just downloaded the app and this can easily be achieved with Interactable. Let me know what you've done so far

@wibb36 we gave it a shot but couldn't figure out how to add snap points properly. We didn't make much progress that's worth sharing. We'd love to get this going as we need it. I'll take another stab on the weekend.

Still can't figure this one out – I'm not well enough versed with GH and reanimated yet. @wibb36 let me know if you take a stab at it.

This issue appear to be inactive.


[stdlib] Dictionary.values: Replace setter with _modify

This should enable in-place modification of the Values view, which should greatly improve the performance of code like this:
dictionary.values[index] = newValue

https://bugs.swift.org/browse/SR-5258 / rdar://problem/44725826

@swift-ci please smoke test

@swift-ci please smoke benchmark

I think we actually have some benchmarks for this.

Build comment file:
Build failed before running benchmark.

Oops, Dictionary.init(_variant:) has not yet landed with https://github.com/apple/swift/pull/19496

I guess it is slightly weird that assigning to values might affect keys, but would it cause any issues in practice?
dict.values = someOtherDict.values   // same as dict = someOtherDict

It seems technically difficult to verify the identity doesn't change. We could remember the raw bit pattern for it, but it wouldn't be foolproof -- the yieldee could end up releasing it and allocating a new one at the same address.

@swift-ci please smoke benchmark

Oh, right, replacing the values replaces the entire dictionary, so it is safe, if confusing. A debug trap might be useful for a case where someone mistakenly thinks it does something different though.

Build comment file:
Performance: -O

TEST
OLD
NEW
DELTA
RATIO

Improvement

DictionarySwapAt
5076
908
-82.1%
5.59x

DictionarySwapAtOfObjects
49768
9443
-81.0%
5.27x

Performance: -Osize

TEST
OLD
NEW
DELTA
RATIO

Improvement

DictionarySwapAtOfObjects
49987
10099
-79.8%
4.95x

DictionarySwapAt
5253
1328
-74.7%
3.96x

Performance: -Onone

TEST
OLD
NEW
DELTA
RATIO

Improvement

DictionarySwapAtOfObjects
113144
35401
-68.7%
3.20x

DictionarySwapAt
34148
17177
-49.7%
1.99x

How to read the data
The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.
If you see any unexpected regressions, you should consider fixing the regressions before you merge the PR.
Noise: Sometimes the performance results (not code size!) contain false alarms. Unexpected regressions which are marked with '(?)' are probably noise. If you see regressions which you cannot explain you can try to run the benchmarks again. If regressions still show up, please consult with the performance team (@eeckstein).

Hardware Overview
  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

The setter didn't include a trap, so adding it in _modify would technically be a source-breaking change. We should probably make a separate issue for it.


toJSON does not work on lists

orientdb {db=Ice}> update element set data = {"Hello":"World"}
Updated record(s) '1' in 0.024000 sec(s).
orientdb {db=Ice}> select data.toJSON() from Element
----+------+------------------------------------------
|@CLASS|data
----+------+------------------------------------------
0   |null  |{"@type":"d","@version":0,"Hello":"World"}
----+------+------------------------------------------
1 item(s) found. Query executed in 0.015 sec(s).
orientdb {db=Ice}> update element set data = [{"Hello":"World"}]
Updated record(s) '1' in 0.002000 sec(s).
orientdb {db=Ice}> select data.toJson() from Element
----+------
|@CLASS
----+------
0   |null
----+------
1 item(s) found. Query executed in 0.002 sec(s).
I expected the second select query to return a list structure. Also, how do I disable the @type and @version from the standard JSON? The documentation is unclear.

Hi @lukeworthtkbt I'm on it

Hi @lukeworthtkbt
there the problem is that toJson() is a method of ODocument, and lists do not have such a method.
There are a couple of work-arounds for this:
select $current.toJSON() from (select element from V)

select $current.toJSON("fetchPlan:data:-1 *:-2") from element

Now I'm verifying if we can implement a new method for that also on lists

Hi @lukeworthtkbt
I finally managed to fix it in develop branch.
It is considered a new feature, so probably I won't be able to backport it to 2.1.x
I'm closing this, please reopen if you need further help or info
Thanks
Luigi


In UDF should function arguments with no default be not null e.g. someFunc(arg: UUID!) vs someFunc(arg: UUID)

Describe the bug
Currently when a function has an argument with no default, it still shows up as nullable in the introspection schema
For example:
begin;

    create function "someFunc" (arg uuid)
        returns int
        immutable
        language sql
    as $$ select 1; $$;

    select jsonb_pretty(
        graphql.resolve($$
            {
            __type(name: "Query") {
                fields(includeDeprecated: true) {
                    name
                    args {
                      name
                      type {
                        kind
                        name
                        ofType {
                          kind
                          name
                        }
                      }
                    }
                }

            }
            }
        $$)
    );

Returns
                     jsonb_pretty
-------------------------------------------------------
 {                                                    +
     "data": {                                        +
         "__type": {                                  +
             "fields": [                              +
                 {                                    +
                     "args": [                        +
                         {                            +
                             "name": "nodeId",        +
                             "type": {                +
                                 "kind": "NON_NULL",  +
                                 "name": null,        +
                                 "ofType": {          +
                                     "kind": "SCALAR",+
                                     "name": "ID"     +
                                 }                    +
                             }                        +
                         }                            +
                     ],                               +
                     "name": "node"                   +
                 },                                   +
                 {                                    +
                     "args": [                        +
                         {                            +
                             "name": "arg",           +
                             "type": {                +
                                 "kind": "SCALAR",    +     -- CAN THIS BE NON-NULL?
                                 "name": "UUID",      +
                                 "ofType": null       +
                             }                        +
                         }                            +
                     ],                               +
                     "name": "someFunc"               +
                 }                                    +
             ]                                        +
         }                                            +
     }                                                +
 }
(1 row)

given that this change would be from nullable to non-null and is only related to inputs, the change would be non-breaking in the API layer

I'd guess that the reason this doesn't currently happen is because we don't support default arguments https://github.com/supabase/pg_graphql/issues/410
but I think it'd still be possible to check if one exists to determine if the function arg is nullable or not

Yes, non-default args should be non-null.


Add a Material_DoubleSided model with a texture but no normals or tangents

Material_DoubleSided
Fixes #499

Can we verify that Babylon is doing it correctly?

The new model's appearance is in line with the other doublesided test models, so it appears correct to me.


[bug] layer control sometimes does not work

as of e88b9af5659da1878e60f4fc32a1e7320bb52e7d
goto /?templateSelected=Big+Brain+%28Histology%29&parcellationSelected=Grey%2FWhite+matter&previewingDatasetFiles=%5B%7B%22datasetId%22%3A%22minds%2Fcore%2Fdataset%2Fv1.0.0%2Fb08a7dbc-7c75-4ce7-905b-690b2b1e8957%22%2C%22filename%22%3A%22Overlay+of+data+modalities%22%7D%5D
sometimes, layer control do not work (see screenshot)

cannot reproduce reliably.

cannot seem to reproduce

cannot reproduce.


Higher Order Operator Overloading

Higher Order Operator Overloading (HOOO) is very important for higher order reasoning. For reference, see papers on path semantics notation https://github.com/advancedresearch/path_semantics/blob/master/sequences.md#notation
By adding HOOO support to Dyon, it could open up a new functional programming paradigm.
I suggest that HOOO should inline by default.
Algebra With Closures
The syntax for "algebra with closures":
a := \(x: f64) = x + 1
b := \(x: f64) = x + 2
c := a + b

This is the same as:
a := \(x: f64) = x + 1
b := \(x: f64) = x + 2
c := \(x: f64) = {
    a := grab a
    b := grab b
    \a(x) + \b(x)
}

The above is inlined to this:
a := \(x: f64) = x + 1
b := \(x: f64) = x + 2
c := \(x: f64) = (x + 1) + (x + 2)

Higher Order Calls
Higher Order Calls are supported by external functions that return some value:
a := \(x: f64) = x + 1
b := exp(a)

This is the same as:
a := \(x: f64) = x + 1
b := \(x: f64) = {
    a := grab a
    exp(\a(x))
}

The above is inlined to this:
a := \(x: f64) = x + 1
b := \(x: f64) = exp(x + 1)

When calling a loaded function, closures are accepted as normal parameters:
fn main() {
    a := \(x: f64) = x + 1
    println(foo(a)) // prints `\(x: f64) = exp(x + 1)
}

foo(x: f64) = exp(x)

The type checker "believes" that closures returning f64 are acceptable as f64 arguments.

In the “higher order calls” example, you wanted to write:

a := \(x: f64) = x + 1
b := \(x: f64) = {
    a := grab a
    exp(\a(x))  // missed the exp here
}

@flying-sheep Thanks!

I believe that Higher Order Calls becomes too complicated.

Type checking becomes messy
Overhead when calling external functions
Can't pass closures to external functions

I've moved Higher Order Calls to a separate issue: https://github.com/PistonDevelopers/dyon/issues/615

I believe that HOOO is too complex in Dyon and instead a language should be designed from ground up to be used with HOOO. It is more suitable for theorem proving than normal programming.
Closing.


enumerating /nix/store/... paths fails

Currently nix builds don't work. ls operations fail during builds.
To reproduce, execute nix-portable inside a debian docker container:
docker run --rm -v $(pwd)/nix-portable:/nix-portable -it debian

inside the container execute any build:
/nix-portable nix build --impure --expr "(import <nixpkgs> {}).hello.overrideAttrs (_:{abc=1;})"

It fails with:
error: builder for '/nix/store/msj4z165q92hgik21g6r2gp6chy9wjyn-hello-2.10.drv' failed with exit code 1;
       last 10 log lines:
       > patching sources
       > configuring
       > configure flags: --disable-dependency-tracking --prefix=/nix/store/p83nw19z0437nfsfh34lyi01lnvn9yy2-hello-2.10
       > checking for a BSD-compatible install... /nix/store/0y7rv26ffa359wvqd3js94cn73z882fg-coreutils-8.32/bin/install -c
       > checking whether build environment is sane... ls: cannot access './configure': No such file or directory
       > ls: cannot access 'conftest.file': No such file or directory
       > configure: error: ls -t appears to fail.  Make sure there is not a broken
       >   alias in your environment
       > configure: error: newly created file is older than distributed files!
       > Check your system clock
       For full logs, run 'nix log /nix/store/msj4z165q92hgik21g6r2gp6chy9wjyn-hello-2.10.drv'.

Solved since version 001


Drawcustum extended with (QR) images

I would like to be able to send the wifi guest QR image to a tag, preferably inside the drawcustom (add other info as well). Might also be extended to other images available from home assistant.

closing it now, will be implemented in a release later today


Implement Renderer Factory

Remove some cycles discovered by Sonar.

coverage: 100.0%. remained the same
when pulling 3ddf8bd601ac29dede66c4d2b521c79e54eb2b73 on jeffgbutler:renderer-factory
into c96a12f200548446e3dac6054ea40251aa170584 on mybatis:master.


RecTec Pellet Smoker Controller Support

Clach04, first let me say thank you for all the work you've put into python-tuya; it's already been incredibly helpful for me.  I recently acquired the new WiFi controller for my RecTec pellet smoker and after some research I found it leverages the Tuya platform which led me to both your project and Tuyapi.  So far I've been able to leverage your status function to get some good info from the smoker (state, temperatures, etc.) however I haven't been able to change its state (i.e. turn it on or off) with the set function (which I realize is designed for other devices - I was just hoping I might get lucky).  I was hoping you might have some insight into how I might be able to do so (or if I'll need to figure out how to leverage the MQTT interface instead).  The scripts I've been working on so far are available here.  I hope you don't mind, but I've added your discover devices script as it greatly simplifies finding the device ID.

Can you post the schema for the device?
On Tue, Jul 17, 2018 at 12:07 AM, SDNick484<EMAIL_ADDRESS>wrote:

Clach04, first let me say thank you for all the work you've put into
python-tuya; it's already been incredibly helpful for me. I recently
acquired the new WiFi controller for my RecTec pellet smoker and after some
research I found it leverages the Tuya platform which led me to both your
project and Tuyapi. So far I've been able to leverage your status function
to get some good info from the smoker (state, temperatures, etc.) however I
haven't been able to change its state (i.e. turn it on or off) with the set
function (which I realize is designed for other devices - I was just hoping
I might get lucky). I was hoping you might have some insight into how I
might be able to do so (or if I'll need to figure out how to leverage the
MQTT interface instead). The scripts I've been working on so far are
available here https://github.com/SDNick484/rectec_status/. I hope you
don't mind, but I've added your discover devices script
https://github.com/codetheweb/tuyapi/issues/5#issuecomment-348799963 as
it greatly simplifies finding the device ID.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/clach04/python-tuya/issues/30, or mute the thread
https://github.com/notifications/unsubscribe-auth/ADLj6DxlKSR-W8u7yV7XSZc-r4LBAL3qks5uHY04gaJpZM4VSS7z
.

@SDNick484 - that is awesome! It fills me with so much joy to know that someone is using something I wrote to  :fire: burn  :fire: things! :grinning: The Alexa skill is a really nice addition.
reverse engineered some information already and posted it in your readme https://github.com/SDNick484/rectec_status/ - As @BillSobel posted, getting the device schema will help grealy here. This would mean taking a different app to register the grill (temporarily) to dump the schema. See https://github.com/clach04/python-tuya/wiki/DeviceSchemas
What have you tried already for turning on/off?
The other avenue to explore would be to trace the official app using tcpdump and then decrypt the packets (you may already have seen the code I posted code for that). I recommend holding off on this but I'm not sure when I'll get chance to respond so dumping what I know during my lunch break, the following should work:
http://www.tcpdump.org/tcpdump_man.html (you may need root, been a while since I did this):
 exec tcpdump -i any -w /sdcard/tcpdump.out  host  grill.ip.address.here

RE MQTT - I do NOT think that is an option. I don't think any of my devices support local MQTT. From what I can gather that is done via the cloud service (which this project does not have support for).
No problem bundling the discover code. At some point it would be nice to get that integrate into the library (PR's welcome :wink:).

Thank you both for the quick responses; I will work on getting the schema tonight.  I've only had the new controller for a week so I haven't had a chance to do much exploration (yet!) however I have already taken a couple tcpdumps during a couple cookouts.  I'll get those posted too.  Haven't tried registering the device with other apps, but that was on my to-do list when I get some time.

Googled this real quick and found the below project, they realized its
yours, lol.  Looks like DP1 is the on/off control...  If you can give me
the product key you mention being broadcast, I can look up the full schema
(the below is a partial schema, not the full tuna one which are more
descriptive).  I have Tuya developer keys and can call the cloud API, the
productKey is a shared key for all devices of the same type.
Bill

Hi Bill, the productKey is "1CwNbNLE8r70WR7t".  I was able to get the schema and will post it on the product page shortly.  I am also working on getting it formatted correctly but am having a little trouble.  I used the Packet Capture + TuyaSmart app method.

@impala454 I am getting back into my project and will try to take a look soon to see what's cha nged. You could definitely get temp data previously over local TCP previously (although you are correct that the app uses MQTT data from the cloud for that). I am hoping to get some Home Assistant integration in place but am pressed for time. Will hopefully have a chance to make some progress in the next few weeks.

@impala454 I just checked my code and it's still working for me on my RT-680 WiFi controller.  I'm wondering if the Stampede is using a newer chip/protocol.
Does this script work for you: https://github.com/codetheweb/tuyapi/issues/5#issuecomment-348799963

@SDNick484 am unable to test at the moment but will try soon.  I'm very interested in running this to ground, as RecTec support has not been forthcoming in trying to get me any data.

@SDNick484 the script you pasted is nearly identical to what I've been trying, and no dice.  I don't see anything on port 6666.  When I wireshark the grill I still just see the packets on 6667 I mentioned above, which appear to be encrypted.  Here's a sample (note that the contents do not change, even with the grill on):
000055aa0000000000000013000000ac00000000aa613a3bdc1e4a5a5c8504d88beb4c44b7fcece0445a1705da4358c12c063fc90fce36c6c7e17c13b3f3db89d740c01ec77675ec78d575e4d20f57eda05ae5bfcb5924bb8792fe5eae7c9040e5921a98017f2170860421e6fc66f0a1ade138915fde850cd157270c5f3d1ab5ae90b300974ebcfdbac62e220d35be537c43de671c8343bc25f3403c0c4c3db59350d34ae0bd1f2cbad705886d4ed8eed5ab9eb9475641140000aa55

Another note, the TCP traffic is definitely TLS encrypted, and going to AWS at regular intervals, or quick bursts while I'm using the app.

Interesting, yeah, they must have made changes as that's definitely not what I'm seeing.   Would be interesting to know which models/controllers that affects.  I'll see if any of my other devices are on the newer firmware and will try to do some digging.  If you haven't already, I'd recommend checking out this thread: https://github.com/codetheweb/tuyapi/issues/175

Minor updates:  I tried the following and it seems I get nothing back until I change version to 3.3.  So I'm guessing that's the version my grill has.  Note the return is encrypted or at least serialized in some way and unreadable.
const TuyAPI = require('tuyapi');

const device = new TuyAPI({
  key: 'xxxxxxxxxxxxxxxx',
  ip: '<IP_ADDRESS>',
  version: 3.3});

(async () => {
  await device.find();

  await device.connect();

  let status = await device.get();

  console.log(`Current status: ${status}.`);

  device.disconnect();
})();

Another update is I was able to de-compile the RecTec app and retrieve two interesting values:  TUYA_SMART_APPKEY and TUYA_SMART_SECRET from the AndroidManifest.xml file. I've attempted to use the latest tuya-cli link tool to connect to the grill with no success.


@-mentions are not auto completing

When I try to do an @-mention, the username is not being auto-completed. Nothing specifically happens when I try @ and I have seen before getting a drop-down of auto-completed usernames. Any reason this would suddenly stop working? I am on the latest 1.3.7. I do not see any logs at all regarding this.

I think this is on my end. Other users are not reporting the issue

Closing this issue. it was just a local cache problem, false alarm :-p


pandoc-types 1.19

pandoc-types-1.19 (John MacFarlane<EMAIL_ADDRESS>@jgm) is out of bounds for:

[ ] BlogLiterately-<IP_ADDRESS> (>=1.16 && < 1.18). Brent Yorgey<EMAIL_ADDRESS>@byorgey. @byorgey. Used by: library
[ ] pandoc-1.19.1 (==1.17.*). John MacFarlane<EMAIL_ADDRESS>@jgm. @jgm. Used by: library, executable, test-suite
[ ] pandoc-citeproc-0.10.3 (>=1.16 && < 1.18). John MacFarlane<EMAIL_ADDRESS>@jgm. @jgm. Used by: library, executable, test-suite
[ ] texmath-0.9 (>=<IP_ADDRESS> && < 1.18). John MacFarlane<EMAIL_ADDRESS>@jgm. @jgm. Used by: library

pandoc-types 1.19 is a bit of a mistake.  It is, in fact,
the same API as pandoc-types-<IP_ADDRESS>.  It reverts some
changes in 1.18, which I'd mistakenly thought I'd already
pushed to Hackage.
Upshot is that it's not going to hurt for packages to
change their bounds to include it, but there's absolutely
no reason to do so.
Indeed, I could deprecate it on Hackage, if this would
make things easier for others.
+++ Michael Snoyman [Jan 22 17 01:55 ]:

pandoc-types-1.19 (John MacFarlane<EMAIL_ADDRESS>[2]@jgm) is out
of bounds for:
* [ ] BlogLiterately-<IP_ADDRESS> (>=1.16 && < 1.18). Brent Yorgey
<EMAIL_ADDRESS>[4]@byorgey. [5]@byorgey. Used by: library
* [ ] pandoc-1.19.1 (==1.17.*). John MacFarlane<EMAIL_ADDRESS>[7]@jgm. [8]@jgm. Used by: library, executable, test-suite
* [ ] pandoc-citeproc-0.10.3 (>=1.16 && < 1.18). John MacFarlane
<EMAIL_ADDRESS>[10]@jgm. [11]@jgm. Used by: library,
executable, test-suite
* [ ] texmath-0.9 (>=<IP_ADDRESS> && < 1.18). John MacFarlane
<EMAIL_ADDRESS>[13]@jgm. [14]@jgm. Used by: library
—
You are receiving this because you were mentioned.
Reply to this email directly, [15]view it on GitHub, or [16]mute the
thread.
References

<EMAIL_ADDRESS>https://github.com/jgm
<EMAIL_ADDRESS>https://github.com/byorgey
https://github.com/byorgey
<EMAIL_ADDRESS>https://github.com/jgm
https://github.com/jgm
<EMAIL_ADDRESS>https://github.com/jgm
https://github.com/jgm
<EMAIL_ADDRESS>https://github.com/jgm
https://github.com/jgm
https://github.com/fpco/stackage/issues/2223
https://github.com/notifications/unsubscribe-auth/AAAL5B6ThmSie-s3frZkVutFG7B836eEks5rUyeGgaJpZM4LqVi0

It looks like I'm the only other person this affects, and I don't particularly care either way.  Bumping the upper bound for pandoc-types in BlogLiterately would be easy enough.  So, what's the right way to fix this?  Anyone have an opinion? @snoyberg ?

With my Stackage curator hat on: it's convenient to be able to remove all upper bounds, and just depend on the most recent version of packages. But keeping an upper bound for a well-documented case like this is not a big deal. So feel free to leave it as-is.
I can't speak authoritatively for the Hackage team, but my guess would be doing a hard deprecation of that version on Hackage (by both deprecating, and editing the cabal file to have an impossible version constraint on a dependency) would be best.

+++ Michael Snoyman [Jan 24 17 03:29 ]:

I can't speak authoritatively for the Hackage team, but my guess would
be doing a hard deprecation of that version on Hackage (by both
deprecating, and editing the cabal file to have an impossible version
constraint on a dependency) would be best.

I can do that, but can you explain what you meant by

editing the cabal file to have an impossible version
constraint on a dependency) would be best.

Sure. I don't remember what "best practices" is right now, but as an example you could use the Hackage revision capabilities to set:
build-depends: base < 4 && base > 4

Since it's impossible for any version of base to fulfill this constraint, the dependency solver will never select it. This works around the fact that deprecating a version of a package is not always respected by the solver (IIRC it's a "soft" vs "hard" constraint).

.... or just base <0 which is less arbitrary.

Just to be clear, if you go to the Hackage page for the package and scroll down to "Maintainer's corner" and click on "edit package information", you can edit the cabal file without uploading a new version.

Done.
+++ Michael Snoyman [Jan 24 17 04:58 ]:

Sure. I don't remember what "best practices" is right now, but as an
example you could use the Hackage revision capabilities to set:
build-depends: base < 4 && base > 4
Since it's impossible for any version of base to fulfill this
constraint, the dependency solver will never select it. This works
around the fact that deprecating a version of a package is not always
respected by the solver (IIRC it's a "soft" vs "hard" constraint).
—
You are receiving this because you were mentioned.
Reply to this email directly, [1]view it on GitHub, or [2]mute the
thread.
References

https://github.com/fpco/stackage/issues/2223#issuecomment-274796321
https://github.com/notifications/unsubscribe-auth/AAAL5JiWJtKMWxH_OFqkRI-YBxznM-vSks5rVfVxgaJpZM4LqVi0


How to send custom adb commands through your server?

Hello developers, I am new to scrcpy, this is a great application because of the speed of its communication with android devices. I am using c # to automate android.
I am wanting to go through a scrcpy server to improve the speed of sending custom commands to an android device. I finished starting up the scrcpy server from the c# code associated with the SharpAdbClient library.
My problem is not knowing how to send custom adb commands. For example: uiautomator dump/dev/tty and get the device's xml layout. or send touch commands, swipe in custom position. or adb command to take individual screenshots (only 1 image at a time) ...
Is there any example code or documentation to do the above? any example code or documentation is appreciated. Thanks a lot.

@rom1v Can you help me?

More details here: https://github.com/Genymobile/scrcpy/issues/231#issuecomment-414111753
Basically, scrcpy does not receive adb commands, but custom commands, which are reimplemented in java (using reflection to access Android framework).
More documentation:

https://github.com/Genymobile/scrcpy/blob/master/DEVELOP.md#hidden-methods
https://github.com/Genymobile/scrcpy/blob/master/DEVELOP.md#input-events-injection
https://blog.rom1v.com/2018/03/introducing-scrcpy/

It is possible to send adb command to get the ui dump while scrcpy is running.

Run open_a_terminal_here.bat to open an another command prompt. (inorder to use same adb.exe as scrcpy)
Type any adb command to do whatever you want.

Get the ui dump.

adb exec-out uiautomator dump && adb pull /sdcard/window_dump.xml && adb shell rm /sdcard/window_dump.xml

Take a screen shot

adb shell screencap -p /sdcard/capture.png && adb pull /sdcard/capture.png && adb shell rm /sdcard/capture.png
It worked on my Android 9 environment.
@rom1v Thank you very much for sherering such a wonderful tool by the way.


autohotkey: add version 2.0-beta.3

Adds AutoHotkey v2 beta
I've copied&pasted the main AutoHotkey manifest, updated all the links to version 2 (with a nice bonus of not relying on GitHub) and tested via both installing 2.0-beta.2 and 2.0-beta.3 and autoupdating 2.0-beta.2 manifest to 2.0-beta.3 with checkver
To avoid conflict with v1 I've added a rename rule AutoHotkey 32/64 .exe→autohotkey2.exe to install a single executable without installing the original 32/64, so you could have

autohotkey 32/64 .exe and autohotkey.exe from autohotkey.json manifest and
autohotkey2.exe from this manifest,

though I guess in some distant future it should just override the first version and manifests be merged into one. Or maybe not since it's worthwile keeping both versions installed as you'll see scripts using both syntaxes
Closes #7446

/verify

autohotkey.com says version 1.0 is deprecated.
I think we should just update the autohotkey package to v2, and move v1 to the versions bucket.
Then, for bin, use autohotkey instead of autohotkey2.
shortcuts is missing.

autohotkey.com says version 1.0 is deprecated.

yeah, this is the old-old version, but it isn't used, current AutoHotkey manifest uses version 1.1, which is very much alive
The new version is 2.0, which entered beta

shortcuts is missing.

so it is for autohotkey.json, and that's supposed to be like that since you can't really launch a portable AHK and expect it to do anything, it just shows a help file, you must supply a path to script (not sure, but I think only AHK installed to Program Files automatically looks for a script in user's documents). So a shortcut that doesn't do anything useful isn't useful

For bin, I think use autohotkey is better than autohotkey2.

but then you wouldn't be able to install both, right?, which is of a significant value during the transition between syntaxes.
Now, I'd personally use autohotkey.exe for version 2 as you suggest (or even ahk.exe for simpler cli typing when you need to check a script or two) and then rename the current manifest with version 1.1 to autohotkey1.exe, but given that most of people are still using version 1.1 not sure that'd be the best approach for them.
What do you think?

I vote for autohotkey2, as many people might not currently know about it because of it's beta status

but then you wouldn't be able to install both, right?, which is of a significant value during the transition between syntaxes.

I see your point. Using autohotkey2 is probably the best option for now.


Fix typo

What is the purpose of the change

wget is written twice when downloading genesis file for the testnet.

Brief change log

Remove wget in this section

Verifying this change
This change has been tested locally by rebuilding the doc website and verified content and links are expected

Thank you!


Removed unneeded TokenUser.ne method.

This method is not needed since Python 3 is the minimum requirement.

#241 should take care of it.


Odd Bundle Inconsistencies

Let me first start off saying again that I love what ansible-silo is doing, it is solving a need for my employer that I haven't been able to find anywhere else.
That being said, I encountered a really odd error yesterday and I can't seem to figure out what is going on.
I've built a bundle and it can run just fine on both Ubuntu 16.04 and Centos 7 machines, however when trying to execute on a RHEL 7 machine, I'm getting this error when trying to install it:
[bl839s@bld-apic-em-1 ~]$ docker run --interactive --tty --rm --volume ~/bin:/silo_install_path qosiga:0.0.14 --install
env: can't execute 'bash': No such file or directory

Showing the RedHat release on this server:
[bl839s@bld-apic-em-1 ~]$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.4 (Maipo)

However, on my development box which is running Centos 7, it works fine:
[bl839s@docker-node4 ~]$ docker run --interactive --tty --rm --volume ~/bin:/silo_install_path qosiga:0.0.14 --install
Installing from qosiga 0.0.14:
 - qosiga will be updated
Done
[bl839s@docker-node4 ~]$

Showing the Centos release on this server:
[bl839s@docker-node4 ~]$ cat /etc/centos-release
CentOS Linux release 7.4.1708 (Core)

Just for testing, I tried installing the plain ansible-silo on the RHEL7 machine and it worked fine:
[root@bld-apic-em-1 bl839s]# docker run --interactive --tty --rm --volume "/usr/local/bin:/silo_install_path" grpn/ansible-silo:2.1.0 --install
Unable to find image 'grpn/ansible-silo:2.1.0' locally
2.1.0: Pulling from grpn/ansible-silo
88286f41530e: Pull complete
3f37066d99ce: Pull complete
2315bb9a4578: Pull complete
dc833ec4b91e: Pull complete
3d6ccaeed476: Pull complete
e1a4e6af9c79: Pull complete
1d1a6cf92652: Pull complete
Digest: sha256:387139cd500cec4c0d5c396c5b1a0f86616427b71d0a1bde1241fcd3dbbc3ca8
Status: Downloaded newer image for grpn/ansible-silo:2.1.0
Installing from ansible-silo 2.1.0:
 - ansible will be created
 - ansible-config will be created
 - ansible-console will be created
 - ansible-doc will be created
 - ansible-galaxy will be created
 - ansible-lint will be created
 - ansible-playbook will be created
 - ansible-pull will be created
 - ansible-silo will be created
 - ansible-vault will be created
Done

Showing it runs just fine too:
[bl839s@bld-apic-em-1 silo]$ ansible-silo --version
ansible-silo 2.1.0
ansible <IP_ADDRESS>
ansible-lint 3.4.20
ansible installed on volume silo.bl839s

This means that there is some inconsistency with the bundle I've built off of ansible-silo, and I'm wondering if you can give any pointers for where to look as to what could be causing this issue?
Any help you can provide would be appreciated.
Thanks again for all of the help given so far and I apologize for continuing to come back with problems, but hopefully it will just help ansible-silo become better in the process.

Good morning Brady,
that's very strange. But I don't think the problem lies in the image/container, there really is not much that could go wrong.
I see the 1st install command of the bundle was executed as user bl839s and the 2nd install command of ansible-silo was executed as root. I'm guessing there is a difference in the env and user bl839s doesn't have bash in the PATH.
Can bl839s call bash? What do those command show?
/usr/bin/env bash --version

command -v bash

The latter command should show the location of bash. Is the parent directory in $PATH?
Compare:
echo $PATH
sudo echo $PATH

Unfortunately this is the only pointer I can give for now. Again, really strange.

Thanks again for all of the help given so far and I apologize for continuing to come back with problems, but hopefully it will just help ansible-silo become better in the process.

Exactly! So don't worry, you're always welcome and I'm happy silo is being used out there.
Cheers,
Daniel

Yeah, I tried installing the bundle both my user and as root with the same results.
I thought the same as well from your comment:

But I don't think the problem lies in the image/container, there really is not much that could go wrong.

Here is the output of the requested commands:
[bl839s@bld-apic-em-1 ~]$ /usr/bin/env bash --version
GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

[bl839s@bld-apic-em-1 ~]$ command -v bash
/usr/bin/bash

[bl839s@bld-apic-em-1 ~]$ echo $PATH
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

[bl839s@bld-apic-em-1 ~]$ su
Password:
[root@bld-apic-em-1 bl839s]# echo $PATH
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

I'm not an expert in Linux, but from what I can tell, it should work fine, something is apparently off though.
I'll keep working on it as much as I can and let you know when and if I come up with anything.
Thanks.

Still trying to troubleshoot this same issue...
I know you're not able to troubleshoot docker problems, but I thought I would ask just in case.
After trying all sorts of things, I just tried installing ansible-silo again per the instructions on your GitHub page and I encountered this error:
[root@bld-apic-em-1 bl839s]# docker run --interactive --tty --rm --volume "/usr/local/bin:/silo_install_path" grpn/ansible-silo:2.1.0 --install
standard_init_linux.go:185: exec user process caused "no such file or directory"

My GoogleFu has been pointing to incorrect line endings (windows causes issues), but I haven't touched anything on this system with Windows.
I'm using a fairly recent version of docker:
[root@bld-apic-em-1 bl839s]# docker --version
Docker version 17.09.1-ce, build 19e2cf6

Which at a basic level seems to work fine:
[root@bld-apic-em-1 bl839s]# docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
ca4f61b1923c: Pull complete
Digest: sha256:083de497cff944f969d8499ab94f07134c50bcf5e6b9559b27182d3fa80ce3f7
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
 1. The Docker client contacted the Docker daemon.
 2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
    (amd64)
 3. The Docker daemon created a new container from that image which runs the
    executable that produces the output you are currently reading.
 4. The Docker daemon streamed that output to the Docker client, which sent it
    to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
 $ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
 https://cloud.docker.com/

For more examples and ideas, visit:
 https://docs.docker.com/engine/userguide/

But when I try and run this container, I get the same error:
[root@bld-apic-em-1 bl839s]# docker run grpn/ansible-silo:2.1.0
standard_init_linux.go:185: exec user process caused "no such file or directory"

I'm grasping at straws trying to understand why this production system is behaving drastically different than all of my development systems.

Leaving a breadcrumb:  here's the code for the version of docker-ce that you're using:

https://github.com/docker/docker-ce/tree/v17.09.1-ce

If I'm correct in my guess about how docker builds up its distribution, here's the place they set the runc version:

https://github.com/docker/docker-ce/blob/v17.09.1-ce/components/engine/hack/dockerfile/binaries-commits#L6

RUNC_COMMIT=3f2f8b84a77f73d38244dd690525642a72156c64

Which leads to the runc code here:

https://github.com/opencontainers/runc/blob/3f2f8b84a77f73d38244dd690525642a72156c64/libcontainer/standard_init_linux.go#L185

P.S., I had previously figured out that this file was in runc by using github to search for the filename in the docker org:

https://github.com/search?q=org%3Adocker+filename%3Astandard_init_linux.go&type=Code&utf8=✓

That led to this repo:

https://github.com/docker/libcontainer

Which indicates it was moved to here:

https://github.com/opencontainers/runc

I wonder why they don't include the failing command in the error... you can possibly hand-hack the library to that.

Similar to issue #54:
I've been able to solve this by rebuilding grpn/ansible-silo-base:3.0.0 and grpn/ansible-silo:3.0.0.
I'm still working through some issues, but when I'm ready I will submit a PR.


Cancel  pending image loading when tile is out-of-view

TANGRAM VERSION:
All
ENVIRONMENT:
All
TO REPRODUCE THE ISSUE, FOLLOW THESE STEPS:
set network throttling to somethin slow
Start fast pan and zoom , fly-to,
Look at network connections
RESULT:
Tiles that moved out of view still loading, This causing huge network queue and as a result tiles that requested last (most recent and in view) loaded after tiles that does not required for scene rendering, also blocking some interactions
EXPECTED RESULT:
not in the view network requests should be canceled (see mapbox for examples)

Any plans about this?


Segmentation fault in <IP_ADDRESS>

clickhouse segmentation fault, unknown reason.

(version <IP_ADDRESS>)
2019.08.25 23:02:12.443223 [ 9442 ] {}  BaseDaemon: ########################################
2019.08.25 23:02:12.443428 [ 9442 ] {}  BaseDaemon: (version <IP_ADDRESS>) (from thread2893) Received signal Segmentation fault (11).
2019.08.25 23:02:12.443441 [ 9442 ] {}  BaseDaemon: Address: NULL pointer.
2019.08.25 23:02:12.443448 [ 9442 ] {}  BaseDaemon: Access: read.
2019.08.25 23:02:12.443454 [ 9442 ] {}  BaseDaemon: Unknown si_code.
2019.08.25 23:02:12.708359 [ 9442 ] {}  BaseDaemon: 0. clickhouse-server(DB::MergeTreeDataPart::getColumnSizeImpl(std::string const&, DB::IDataType const&, std::unordered_set<std::string, std::hashstd::string, std::equal_tostd::string, std::allocatorstd::string >*) const+0xa5) [0x63138b5]
2019.08.25 23:02:12.708417 [ 9442 ] {}  BaseDaemon: 1. clickhouse-server(DB::MergeTreeDataPart::getTotalColumnsSize() const+0xaf) [0x6313a2f]
2019.08.25 23:02:12.708429 [ 9442 ] {}  BaseDaemon: 2. clickhouse-server(DB::StorageSystemParts::processNextStorage(std::vector<COWDB::IColumn::mutable_ptrDB::IColumn, std::allocator<COWDB::IColumn::mutable_ptrDB::IColumn > >&, DB::StoragesInfo const&, bool)+0xa36) [0x5edc6e6]
2019.08.25 23:02:12.708440 [ 9442 ] {}  BaseDaemon: 3. clickhouse-server(DB::StorageSystemPartsBase::read(std::vector<std::string, std::allocatorstd::string > const&, DB::SelectQueryInfo const&, DB::Context const&, DB::QueryProcessingStage::Enum, unsigned long, unsigned int)+0xbd) [0x5ee15cd]
2019.08.25 23:02:12.708451 [ 9442 ] {}  BaseDaemon: 4. clickhouse-server(DB::InterpreterSelectQuery::executeFetchColumns(DB::QueryProcessingStage::Enum, DB::InterpreterSelectQuery::Pipeline&, std::shared_ptrDB::PrewhereInfo const&, std::vector<std::string, std::allocatorstd::string > const&)+0x1e4b) [0x607b69b]
2019.08.25 23:02:12.708470 [ 9442 ] {}  BaseDaemon: 5. clickhouse-server(DB::InterpreterSelectQuery::executeImpl(DB::InterpreterSelectQuery::Pipeline&, std::shared_ptrDB::IBlockInputStream const&, bool)+0x742) [0x607ec82]

It looks like it's connected to some query from system.parts. Can you try to find that query in /var/log/clickhouse-server/clickhouse-server.log ? It should be somewhere before the segfault.

It looks like it's connected to some query from system.parts. Can you try to find that query in /var/log/clickhouse-server/clickhouse-server.log ? It should be somewhere before the segfault.

I check the log and found,
The part /clickhouse/tables/shard/nx_access/replicas/clickhouse6-master/parts/20190726_100844_101049_44 on a replica suddenly appeared, will recheck checksums
And check system.query_log exist like parts query,
select database, table, sum(bytes) as bytes, count() as parts, sum(rows) as rows from system.parts where active = 1 group by database, table
But, this query not cause segfault/crash.

This bug is fixed in https://github.com/yandex/ClickHouse/pull/6514 and backported to 19.13, 19.11.


Not working

cannot be compiled.
always like this: https://prnt.sc/uh4j1e

🤷 Hi, i can't help you with this only screen

🤷 Hi, i can't help you with this only screen

you have discord?

are you trying to obfuscate a big file ? and no i don't have discord

are you trying to obfuscate a big file ? and no i don't have discord

6 mb file

that why he doesn't work u can only obfuscate lower size file

that why he doesn't work u can only obfuscate lower size file

the size is not big. medium size :/ and What should I do now?

try another obfuscator confuserEx or NetGuard

try another obfuscator confuserEx or NetGuard

it unpack easily


Some minor fixes

Fix issues in POM files not being linked together.

Add missing widget set in demo project.

Address some warnings.

Still need more tweaking. I'll reopen when I'm done. Noticed the packages don't match the one released

Issues fixed.

Successfully used it as dependency on a Vaadin project as version 2.0.2-SNAPSHOT


PreviewHandler not works for system know file extension(.cs,.txt and so on)

when i using PreviewHandler, i found it can not do preview for system known file extension, for example, .cs, .txt, anyone who have ever encountered this issue? or anyone who know how to fix this issue?

Below image is my code snip,

Hey @akaksohot. Can you first please change the issue-description according to the template for consistency (see the template when creating a new issue)? Also, depending on which version of SharpShell you are using, this could explain the issue.
Please check the registry yourself according to https://github.com/dwmkerr/sharpshell/issues/229#issuecomment-435536487.
Maybe the "Class" is missing in the registry, which should be fixed in the most current version of the project.
Also you could try to use AssociationType.FileExtension and see if that works (even if this is deprecated, which resulted in issues before, see https://github.com/dwmkerr/sharpshell/issues/283).

Hi Countryen,
Very thanks for your quick reply. SharpShell functions are very great.
My ShareShell is <IP_ADDRESS>, window version is win10.
What i want to do is to make all files and directory have my same custom preview window using SharpPreviewHandler. So i set as below, but it does not work for some system extension which windows system have it's own preview content.
[COMServerAssociation(AssociationType.AllFilesAndFolders)]
[COMServerAssociation(AssociationType.AllFiles)]
Based on above issue, i have tried to add for .cs and .txt, and remove AssociationType.AllFilesAndFolders and AssociationType.AllFiles, as below, it will work for .cs and .txt, but i think many kinds of file have it's own preview content(from windows system), i cannot add for every kind of file. So how do i make every kind of file have my custom preview content by SharpPreviewHandler.
[COMServerAssociation(AssociationType.ClassOfExtension, ".cs")]
[COMServerAssociation(AssociationType.ClassOfExtension, ".txt")]
Will Attach more image in next comment.

Below is the images.

Below is my code.
using System.Runtime.InteropServices;
using System.Windows.Forms;
using SharpShell.Attributes;
using SharpShell.SharpPreviewHandler;
namespace AbcPreviewHandler
{
///
/// The Abc Preview Handler is a preview handler that shows a simple
/// coloured background for ABC files.
///
[ComVisible(true)]
//Case ①
//This is work for .cs and .txt, if want to support more kinds of file which windows system have it's prview content, i need to add more ClassOfExtension.
//And I donot know all kinds of these files which windows system have it's prview content
[COMServerAssociation(AssociationType.ClassOfExtension, ".cs")]
[COMServerAssociation(AssociationType.ClassOfExtension, ".txt")]

//Case ②
//Below two line does not work for .cs and .txt which have it's preview content from windows system.
//[COMServerAssociation(AssociationType.AllFilesAndFolders)]
//[COMServerAssociation(AssociationType.AllFiles)]

[DisplayName("Abc Preview Handler")]
[Guid("B86199F1-13D0-4711-AFE6-08C1E4C58905")]
[PreviewHandler(DisableLowILProcessIsolation = false)]
public class AbcPreviewHandler : SharpPreviewHandler
{
    /// <summary>
    /// DoPreview must create the preview handler user interface and initialize it with data
    /// provided by the shell.
    /// </summary>
    /// <returns>
    /// The preview handler user interface.
    /// </returns>
    protected override PreviewHandlerControl DoPreview()
    {
        //  Create the handler control.
        var handler = new AbcPreviewHandlerControl();
        MessageBox.Show(SelectedFilePath);
        //  Do we have a file path? If so, we can do a preview.
        if (!string.IsNullOrEmpty(SelectedFilePath))
        {
            handler.DoPreview(SelectedFilePath);
        }
        //  Return the handler control.
        return handler;
    }
}

}

using System.Runtime.InteropServices;
using System.Windows.Forms;
using SharpShell.Attributes;
using SharpShell.SharpPreviewHandler;
namespace AbcPreviewHandler
{
///
/// The Abc Preview Handler is a preview handler that shows a simple
/// coloured background for ABC files.
///
[ComVisible(true)]
//Case 1
//This is work for .cs and .txt, if want to support more kinds of file which windows system have it's prview content, i need to add more ClassOfExtension.
//And I donot know all kinds of these files which windows system have it's prview content
[COMServerAssociation(AssociationType.ClassOfExtension, ".cs")]
[COMServerAssociation(AssociationType.ClassOfExtension, ".txt")]

//Case 2
//Below two line does not work for .cs and .txt which have it's preview content from windows system.
//[COMServerAssociation(AssociationType.AllFilesAndFolders)]
//[COMServerAssociation(AssociationType.AllFiles)]

[DisplayName("Abc Preview Handler")]
[Guid("B86199F1-13D0-4711-AFE6-08C1E4C58905")]
[PreviewHandler(DisableLowILProcessIsolation = false)]
public class AbcPreviewHandler : SharpPreviewHandler
{
    /// <summary>
    /// DoPreview must create the preview handler user interface and initialize it with data
    /// provided by the shell.
    /// </summary>
    /// <returns>
    /// The preview handler user interface.
    /// </returns>
    protected override PreviewHandlerControl DoPreview()
    {
        //  Create the handler control.
        var handler = new AbcPreviewHandlerControl();
        MessageBox.Show(SelectedFilePath);
        //  Do we have a file path? If so, we can do a preview.
        if (!string.IsNullOrEmpty(SelectedFilePath))
        {
            handler.DoPreview(SelectedFilePath);
        }
        //  Return the handler control.
        return handler;
    }
}

}

Below is my code.
using System.Runtime.InteropServices;
using System.Windows.Forms;
using SharpShell.Attributes;
using SharpShell.SharpPreviewHandler;
namespace AbcPreviewHandler
{
///
/// The Abc Preview Handler is a preview handler that shows a simple
/// coloured background for ABC files.
///
[ComVisible(true)]
//Case 1
//This is work for .cs and .txt, if want to support more kinds of file which windows system have it's prview content, i need to add more ClassOfExtension.
//And I donot know all kinds of these files which windows system have it's prview content
[COMServerAssociation(AssociationType.ClassOfExtension, ".cs")]
[COMServerAssociation(AssociationType.ClassOfExtension, ".txt")]

//Case 2
//Below two line does not work for .cs and .txt which have it's preview content from windows system.
//[COMServerAssociation(AssociationType.AllFilesAndFolders)]
//[COMServerAssociation(AssociationType.AllFiles)]

[DisplayName("Abc Preview Handler")]
[Guid("B86199F1-13D0-4711-AFE6-08C1E4C58905")]
[PreviewHandler(DisableLowILProcessIsolation = false)]
public class AbcPreviewHandler : SharpPreviewHandler
{
    /// <summary>
    /// DoPreview must create the preview handler user interface and initialize it with data
    /// provided by the shell.
    /// </summary>
    /// <returns>
    /// The preview handler user interface.
    /// </returns>
    protected override PreviewHandlerControl DoPreview()
    {
        //  Create the handler control.
        var handler = new AbcPreviewHandlerControl();
        MessageBox.Show(SelectedFilePath);
        //  Do we have a file path? If so, we can do a preview.
        if (!string.IsNullOrEmpty(SelectedFilePath))
        {
            handler.DoPreview(SelectedFilePath);
        }
        //  Return the handler control.
        return handler;
    }
}

}

Hey @akaksohot,
so you want to override the preview for ALL file-extensions that have a certain (system) preview handler?
I don't think it's possible. You need to specify which extensions/classes your handler registers. And, I think, it's by design that a "specific extension" overrites the "All files and folder" ones, by default.
You might wanna do a reg-scan, look for any extension with this specific preview handler, and then register your handler for every type (override).
I'd recommend using a GUI for the user to chose the extensions. Depending on what you want to achieve, the user could want  to chose the preview handler per extension and you shouldn't "enforce" yours.
Maybe there is already an easy way to find "all extensions with preview handler X"?

hi Countryen,
Thanks for your response.
Confirm that if you mean i can not use the Case 2 code above to implement override all files which include system file '.cs' and '.txt' and so on. I have to user Case1 code above to add for each kind of file(inlcude have system preview content and do not have)?

You should be able to combine them, did you try that?
[ComVisible(true)]
[COMServerAssociation(AssociationType.ClassOfExtension, ".cs")]
[COMServerAssociation(AssociationType.ClassOfExtension, ".txt")]
[COMServerAssociation(AssociationType.AllFilesAndFolders)]
[DisplayName("Abc Preview Handler")]
[Guid("B86199F1-13D0-4711-AFE6-08C1E4C58905")]
[PreviewHandler(DisableLowILProcessIsolation = false)]
public class AbcPreviewHandler : SharpPreviewHandler {...}

This should add your handler to * (& Directory & Folder) & .cs & .txt in HKCR.

will try later.
But if have more file type have system preview content, i have to add more attribute for these files? I think this is not convenient for me.

will try later.
But if have more file type have system preview content, i have to add more attribute for these files? I think this is not convenient for me.

Yes, you have to specify them explicitely, even then they might not work as there is no guarantee that your handler will override the others.
You can also remove the attributes and just write your own Register-/Unregister-Handler, if that is more convenient to you.
As said, I recommend searching the registry for all the extensions/types.

how to search the registry, using what key word or how to search? could you please give some advise on this? Thanks in advance again for your help and response.

There are multiple tutorials/how-to online, here a few:

https://dotnet-snippets.de/snippet/using-the-registry/216
https://dotnet-snippets.de/snippet/nuetzliches-aus-der-registry/428
https://www.infoworld.com/article/3073167/how-to-access-the-windows-registry-using-c.html

Including the MSDN / Microsoft Docs:
https://docs.microsoft.com/de-de/dotnet/api/microsoft.win32.registry?view=netframework-4.8
I don't know if there is a "search for X in Y" function, but you could just iterate over all in HKCR and check the keys for yourself. You'd need only to check the ones starting with "." but also the ones with classes for extensions - so maybe better all of them. I think, you also need to check shell and shellex

Will try the combin attributes above and the search registry approach that you approvided later, then will update the result here.
Thanks a lot for your help.

Hey @akaksohot have you figured out a way? I'd be happy to read your solution (maybe others, too). Also, please remember to close this issue, if the actual "problem with sharpshell" has been resolved for you :)

I have tried to combin Case 1 and Case 2 attributes, but not works, only Case 1 will works, Case 2 will not work it means case 2 do not have affect on the file preview content, no preview content for the files type except case 1(.cs,.txt).


Provide more ways to run with context

Motivation:
If RequestContext provides run(Runnable) or run(Callable) we can reduce
the amount of boilerplate code like the following:
try (SafeCloseable ignored = ctx.push()) {
    task.run()
}

Result:
Fixes #2611

Oops, didn't notice @minwoox didn't review this PR. Please feel free to let us know if you have a concern, @minwoox. :sweat_smile:

Oops, didn't notice @minwoox didn't review this PR. Please feel free to let us know if you have a concern, @minwoox. :sweat_smile:

Looks fantastic!


Add korean patch date for 5.0

Ah, thanks for that.
Can you do me a favour and update the Shadowbringers patch date on the stormblood branch as well (link to file)? Most patches don't need updating cross-branch, but xpacs need representation on all active branches.

I have done #673  😀


[BUG] Fine-tuned model outputs are empty.

Describe the bug
Using deepspeed+transformers, I have fine-tuned an already fine-tuned mT5-XXL model (https://huggingface.co/mt5-coref-pytorch/link-append-xxl) to work better on my native language. Running inference using the model from HuggingFace works just fine (without my own fine-tuning). However, when I load the state_dict I saved from my own fine-tuning (also deepspeed+transformers), the model outputs nothing instead. The state_dict is 50gb+, seems to have the right param/elems amount and loads without errors.
I uploaded the outputs of fine-tuning and both pre-finetuning and post-finetuning inference to https://github.com/IYIAscension/MasterThesis after redacting some of it.
To Reproduce
I uploaded the files I used to github, as well as the output:
https://github.com/IYIAscension/MasterThesis
The data is not mine nor open-source, so I only uploaded a small fragment.
The .sh files were used to run the Python files in a SLURM environment
The original_finetuner.sh file uses the model_finetuner.py file.
The original_inference.sh file uses the model_tester.py file.
CHANGE:
cp -r $HOME/original/test_texts "$TMPDIR"
cp -r $HOME/original/first_model.bin "$TMPDIR"

mkdir "$TMPDIR"/riddlecoref_coref_texts

python -m torch.distributed.run --nproc_per_node=2 $HOME/original/model_tester.py "$TMPDIR"/test_texts "$TMPDIR"/riddlecoref_coref_texts "$TMPDIR"/first_model.bin

INTO:
cp -r $HOME/original/test_texts "$TMPDIR"

mkdir "$TMPDIR"/riddlecoref_coref_texts

python -m torch.distributed.run --nproc_per_node=2 $HOME/original/model_tester.py "$TMPDIR"/test_texts "$TMPDIR"/riddlecoref_coref_texts

FOR PRE-FINETUNING INFERENCE
Expected behavior
After loading the state_dict that I saved using deepspeed (zero_to_fp32.py):
state_dict = torch.load(sys.argv[3])
model = T5ForConditionalGeneration.from_pretrained(None, config=model_config, state_dict=state_dict)

the model should output stuff similar to the outputs found in train_examples in the github, or at least something, just like the model loaded directly from HuggingFace.
ds_report output
[2024-06-28 15:30:42,418] [INFO] [real_accelerator.py:203:get_accelerator] Setting ds_accelerator to cuda (auto detect)
^[[93m [WARNING] ^[[0m async_io requires the dev libaio .so object and headers but these were not found.
^[[93m [WARNING] ^[[0m async_io: please install the libaio-devel package with yum
^[[93m [WARNING] ^[[0m If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
^[[93m [WARNING] ^[[0m Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
^[[93m [WARNING] ^[[0m sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.1
^[[93m [WARNING] ^[[0m using untested triton version (2.2.0), only 1.0.0 is known to be compatible
Could not import _export_to_torch_ir. Please make sure your PyTorch version is newer than 2.2.0.
--------------------------------------------------
DeepSpeed C++/CUDA extension op report
--------------------------------------------------
NOTE: Ops not installed will be just-in-time (JIT) compiled at
      runtime if needed. Op compatibility means that your system
      meet the required dependencies to JIT install the op.
--------------------------------------------------
JIT compiled ops requires ninja
ninja .................. ^[[92m[OKAY]^[[0m
--------------------------------------------------
op name ................ installed .. compatible
--------------------------------------------------
^[[93m [WARNING] ^[[0m async_io requires the dev libaio .so object and headers but these were not found.
^[[93m [WARNING] ^[[0m async_io: please install the libaio-devel package with yum
^[[93m [WARNING] ^[[0m If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
async_io ............... ^[[93m[NO]^[[0m ....... ^[[93m[NO]^[[0m
fused_adam ............. ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
cpu_adam ............... ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
cpu_adagrad ............ ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
cpu_lion ............... ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
^[[93m [WARNING] ^[[0m Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
evoformer_attn ......... ^[[93m[NO]^[[0m ....... ^[[93m[NO]^[[0m
fp_quantizer ........... ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
fused_lamb ............. ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
fused_lion ............. ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
inference_core_ops ..... ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
cutlass_ops ............ ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
transformer_inference .. ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
quantizer .............. ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
ragged_device_ops ...... ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
ragged_ops ............. ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
random_ltd ............. ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
^[[93m [WARNING] ^[[0m sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.1
^[[93m [WARNING] ^[[0m using untested triton version (2.2.0), only 1.0.0 is known to be compatible
sparse_attn ............ ^[[93m[NO]^[[0m ....... ^[[93m[NO]^[[0m
spatial_inference ...... ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
transformer ............ ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
stochastic_transformer . ^[[93m[NO]^[[0m ....... ^[[92m[OKAY]^[[0m
--------------------------------------------------
DeepSpeed general environment info:
torch install path ............... ['/sw/arch/RHEL8/EB_production/2023/software/PyTorch/2.1.2-foss-2023a-CUDA-12.1.1/lib/python3.11/site-packages/torch']
torch version .................... 2.1.2
deepspeed install path ........... ['/home/aabdalla/.local/lib/python3.11/site-packages/deepspeed']
deepspeed info ................... 0.14.2, unknown, unknown
torch cuda version ............... 12.1
torch hip version ................ None
nvcc version ..................... 12.1
deepspeed wheel compiled w. ...... torch 2.1, cuda 12.1
shared memory (/dev/shm) size .... 251.76 GB

Screenshots
Please view the .out files on the github page or request screenshots of certain things.
System info (please complete the following information):

OS: Red Hat Enterprise Linux 8.6
GPU count inference: 2
GPU count finetuning: 32 (4 A100 40gb gpus x 8 nodes)
Interconnects: NVlinks
Python version: 3.11.3
backend: nccl
Pytorch version: 2.1.2
Cuda version: 12.1.1
Triton version: 2.2.0

Launcher context
python -m torch.distributed.run was used for both inference and finetuning.
Docker context
Docker is not used for this.
Additional context
The error "FileNotFoundError: [Errno 2] No such file or directory: '/home/aabdalla/.triton/autotune/Fp16Matmul_4d_kernel.pickle.tmp' -> '/home/aabdalla/.triton/autotune/Fp16Matmul_4d_kernel.pickle'" may be the culprit, however, I didn't succeed in getting a triton version lower than 2.2.0 in the HPC environment (iirc it was a sudo permissions issue). Fp16Matmul_4d_kernel.pickle does exist in the folder, but there is also a Fp16Matmul_4d_kernel.pickle.lock file after running finetuning. I'm still not sure if this is the issue since the model does save and load with the correct amount of params/elems. (finished initializing model - num_params = 558, num_elems = 12.92B). If this is the issue, I do not know how to solve it.
I also looked at the outputs of the preprocessor function in the model_finetuner.py file and it seems correct. I also noticed the difference in training and eval loss, but both evaluation and training files are correctly formatted, I do not think this is the issue. The training consisted of 2 epochs of 6308 examples, which should be plenty.
EDIT: I am aware the model_finetuner.py uses 512 max_length and the model_tester.py uses 2048 max_length. To my understanding this should be fine, the last 512 tokens are much more important and are enough training in my use case. I was surprised that a 2048 or 1024 max_length mT5 did not fit on 8 nodes, but this is not a large issue.

To add to this, I noticed the zero_to_fp32.py is generated in the checkpoint directory. I have been using a zero_to_fp32.py downloaded from github. Could this be the issue?

May be wrong, but I currently believe the 2048->512 max_length change combined with the high learning rate made it learn to answer nothing, so I am closing this issue.


ReactJS support for Rickshaw Library

Hi ,
Does Rickshaw JS library support ReactJS development too..? If Yes, is there any example or tutorial link..?

Get a handle on the Rickshaw object in a module with:
import * as Rickshaw from 'rickshaw';
So far I haven't been able to get Rickshaw to access to jQuery however for some more advanced features.


Correspond to Azure OpenAI API

ref. https://weaviate.io/developers/weaviate/modules/retriever-vectorizer-modules/text2vec-openai#provide-the-api-key

X-Azure-Api-Key: YOUR-AZURE-API-KEY for Azure OpenAI, and

Quick question: is :azure_openai a better more descriptive name? Just curious.

Quick question: is :azure_openai a better more descriptive name? Just curious.

I had a feeling of wanting to map with the X-Azure-Api-Key Header (since it might be dynamically mapped in the future), but :azure_openai might be more accurate.

Does the Azure version supported all the same endpoints? Has all the same
API methods and routes?
On Thu, May 18, 2023 at 7:42 PM Moeki Kawakami @.***>
wrote:

Quick question: is :azure_openai a better more descriptive name? Just
curious.
I had a feeling of wanting to map with the X-Azure-Api-Key Header (since
it might be dynamically mapped in the future), but :azure_openai might be
more accurate.
—
Reply to this email directly, view it on GitHub
https://github.com/andreibondarev/weaviate-ruby/pull/18#issuecomment-1553855014,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AIY6R42RSTWXI2RTCXIGNWTXG26Y3ANCNFSM6AAAAAAYHDUICI
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Source Labs LLC
www.sourcelabs.io http://www.sourcelabs.io/
Cell:<PHONE_NUMBER> <(703)%20347-4980>
*LinkedIn: *https://www.linkedin.com/in/andreibondarev/

Yes. The Azure version has complete compatibility, excluding settings.
https://weaviate.io/developers/weaviate/modules/retriever-vectorizer-modules/text2vec-openai#azure-openai-or-openai

The instructions vary slightly based on whether you are using OpenAI directly or Azure OpenAI. Please make sure that you are following the right instructions for your service provider.
The differences are in:

Parameter names used in the schema, and
Names of the API key to be used.

@moekidev I'm just debating how to best integrate it with the Langchain.rb. Do you think we can assume it's in Azure based on the URL? Like does the azure url mention "azure" (or other suitable identifier) in the name?

@moekidev I'm just debating how to best integrate it with the Langchain.rb. Do you think we can assume it's in Azure based on the URL? Like does the azure url mention "azure" (or other suitable identifier) in the name?

Interesting. From the perspective of Langchain.rb, simply calling it azure might be a bit subtle. I'll change it to :azure_openai.

@moekidev Yeah, I think we just add azure_openai: "OpenAI" here: https://github.com/andreibondarev/langchainrb/blob/main/lib/llm/base.rb#LL10C13-L10C13

@moekidev Do you know if setting these parameters is required for the Azure OpenAI?
        "text2vec-openai": {
          "resourceName": "<YOUR-RESOURCE-NAME>",
          "deploymentId": "<YOUR-MODEL-NAME>",

@moekidev Do you know if setting these parameters is required for the Azure OpenAI?
        "text2vec-openai": {
          "resourceName": "<YOUR-RESOURCE-NAME>",
          "deploymentId": "<YOUR-MODEL-NAME>",

Yes. I have tried TypeScript client.

@moekidev Okay, I can merge this PR. If you feel like making the change to the langchain.rb -- I welcome it. If not -- no worries, I can do it.

@moekidev Sorry -- you didn't answer this question: Do you think we can assume it's in Azure based on the URL? Like does the azure url mention "azure" (or other suitable identifier) in the name? Like is the URL in the following format: "openai.azure.microsoft.com" or whatever?

Sorry -- you didn't answer this question: Do you think we can assume it's in Azure based on the URL? Like does the azure url mention "azure" (or other suitable identifier) in the name? Like is the URL in the following format: "openai.azure.microsoft.com" or whatever?

Sorry. the URL is **.openai.azure.com. I think we can assume.

@moekidev I'll revert the previous PR, and can you make the change that just looks at the url and sets the proper header?

OK

@moekidev Thank you!

@andreibondarev I tried it but it didn't work. It seems that Weaviate determines whether the header from the client is X-OpenAI-Api-Key or X-Azure-Api-Key and switches the endpoint accordingly. Therefore, there's a need to switch the header on the client side like this PR.

I think the support for the Azure OpenAI API in Langchain.rb and the support in weaviate-ruby are not related. This PR change is about a module in Weaviate, and when integrating Langchain.rb and Weaviate, we should embed via Langchain.rb and then passing it to Weaviate. That way, we can align with other vector databases and APIs.

I tried it.
https://github.com/andreibondarev/langchainrb/pull/21

@moekidev Somewhere in this method, around this line: https://github.com/andreibondarev/weaviate-ruby/blob/main/lib/weaviate/client.rb#L83
Can we not just do:
if url.match(/openai.azure.com/)
  headers["X-Azure-Api-Key"] = model_service_api_key
else
  headers[API_KEY_HEADERS[model_service]] = model_service_api_key
end

@andreibondarev It's not correct. The url is not **.openai.azure.com. It's always Weaviate database URL. **.openai.azure.com is the URL that Weaviate's text2vec-openai module accesses.
ref:
https://github.com/weaviate/weaviate/blob/192cddb04e08473bc63d7a5335d0a1b506983cfe/modules/text2vec-openai/clients/vectorizer.go#L52-L62

@moekidev Merged again! This gem's version 0.8.2 should already include the your original PR.


Fix docs regarding celery setup

Thanks, but that docs are correct for the current in-development version (master). Closing as invalid.


Remote Mapping

Hello guys, I am kind of new here. I am trying to upload the spatial mapping meshes to a server. Not all at once (as in Remote Mapping) but progressively, only sending the new, updated or the removed mesh data. I think I will have to rewrite the SpatialMappingObserver for this. Maybe you have a better idea how to approach the problem.
Anyway, looking into the code I looked at the SendMeshes() method in RemoteMappingManager.cs:
Is there a reason why the List meshesToSend is inside the foor loop sending only the first element per iteration or am I missing something? I just put the lines 104, 119 & 120 out of the loop, so that all the data is sent over one single tcp connection (only one serialize and one senddata call).

@darax or @angelaHillier may be able to provide some information here.

I might have planned to change that loop into a coroutine if the performance didn't look right.
To your first question, I tossed this together a few weeks ago for a project I'm working on, it doesn't handle deletes, but you could subscribe to the SurfaceChanged event to handle that:
BuildingMapSurfaceObserver.zip

Wow, thank you very much darax, I will definitely take a good look at it!


UX for specifying template parameters is very poor.

Description
When deploying arm templates with parameters, it's necessary to encode the parameter values as JSON documents, which is very cumbersome to do on the command-line, particularly if the parameter values must be retrieved dynamically (e.g., from other CLI commands).
As a thought exercise, consider the experience for deploying a QuickStart template that accepts Azure resource IDs as parameters. (https://raw.githubusercontent.com/Microsoft/openshift-origin/master/azuredeploy.json)
It should be possible to specify template parameters to the deployment create command itself, essentially promoting template parameters to CLI command parameters.
For example,
az group deployment create --template-uri [some uri] --param1fromtemplate [value] --param2fromtemplate value

Maybe, we should just expose an arg accepting key-value pairs list.

@yugangw-msft tag style syntax sounds good to me!  --parameters param1=val1 param2=val2 ...

@yugangw-msft : lets do this.
I am adding this to upcoming sprint for us to pick


Unexpected space insertion in for loop

TypeScript Version:  master branch
Code
for (let i = startIndex; i < parts.length; i++ , startLine++) {
}

Expected behavior:
i++, startLine++
Actual behavior:
i++ , startLine++

Seems fixed in latest.


ID-42: increase password hash strength

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (1509fe8) 76.82% compared to head (525c310) 76.82%.

Additional details and impacted files
@@            Coverage Diff             @@
##             develop      #97   +/-   ##
==========================================
  Coverage      76.82%   76.82%
  Complexity       209      209
==========================================
  Files             33       33
  Lines            630      630
==========================================
  Hits             484      484
  Misses           146      146

Files
Coverage Δ

src/Application/Security/Password.php
87.50% <100.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.