CWL v1.0.1 support

Work in progress, not ready to merge.
Update cwltoil to complete CWL v1.0.1

Now passing CWL conformance tests 100% locally but blocked on a couple of other branches that need to merge first:
CWL conformance test fix: https://github.com/common-workflow-language/common-workflow-language/pull/496
cwltool updates: https://github.com/common-workflow-language/cwltool/pull/516

Thanks for looking at it @mr-c!  I think I see what I did wrong, just silly mistakes.

Jenkins, test this please

Oh yeah, the tests pass! Now to try it out at EBI with LSF :-)

Hi.
I hope not to interfere with the PR process, but I made a PR (#1824) to this branch that contains a couple of unit tests around reading from and writing to directories.
The tests pass on my system, however I have noticed something "strange". One of the key/values of the generated json contain the following
'http://commonwl.org/cwltool#generation': 0,
when generating a directory. I am not sure what that means and the url does not appear to be valid.

@stefanoberri thanks for the PR with additional testing.
The "generation" field is related to a cwltool-specific extension: https://github.com/common-workflow-language/cwltool/issues/382 it shouldn't be leaking out like that, I'll see about adding a pass to remove it from the final output.

Thank you for implementing the directory support!
It looks like that when jenkins run the tests it does not work ;(

Looks like it failed for an unrelated reason: Post https://quay.io/v2/ucsc_cgl/toil-it/blobs/uploads/: net/http: TLS handshake timeout ping @ejacox

Looks like it failed for an unrelated reason: Post https://quay.io/v2/ucsc_cgl/toil-it/blobs/uploads/: net/http: TLS handshake timeout

This is being addressed in #1822.

@tetron Sorry about that extra commit, I meant to put it in another branch as a PR to this one

@tetron I'm still getting a File Literal related error: https://github.com/BD2KGenomics/toil/issues/1836#issue-250580172

@ejacox I've added code documentation and confirmed that Toil in this branch works with bcbio.  I think it's ready for a look by someone on the core Toil team.

+1 This branch looks great to me based on local tests with the bcbio test CWL. Thanks for all this great work Peter, Michael, Edwin and all. Looking forward to testing this at scale on some bigger workflows.

Thank you for the reviews @mr-c and @chapmanb

On AWS, for the GA4GH-DREAM challenges, cwltoil is giving a similar error for bcbio, gdc dna-seq, and biowardrobe (
example_error.txt
) with the same cwltool "failure to evaluate the expression" (using cwltoil).
Also, #1796 using the maxCores option I assume can be addressed by adding this option to the "hints" section recently created in the last commit for pushing the --defaultCores, etc options.

For the Javascript expression issues with cwltool, what node.js do you have installed on your machine? It looks like something is problematic with the setup but Peter might have might idea exactly what.
For --maxCores, we need more of a global metric in Toil rather than task specific changes in the CWL support. We can already specify correctly how many cores/CPU a particular task uses but would like to be able to say something like: Run these 100 tasks each using 8 cores and 32Gb of memory, but only allocate up to 64 total cores at a time. Then Toil would run these in batches for 8 at a time, cycling through the 100. I believe this works cleanly on single machines and AWS but not sure about on batch systems. Thanks much for looking at this.

@chapmanb , it was node.js.  Thank you, I should have seen that.
I appreciate laying out the description.  I was misunderstanding the problem.  I'll try testing on a batch system with the bcbio files.

@DailyDreaming did you get it working?  I agree with @chapmanb that it is related to the use of nodejs to evaluate expressions but I'm not really sure what is going wrong there.

@tetron I did not, but it is giving me a new error now at least:
new_error.txt

It looks like things are running now but there is something problematic with workflow. It's complaining about not finding any outputs but it looks like the command line passes "False" so doesn't actually run the rmdup command. I don't really know much about that workflow, it might be worth asking the authors if they have any pointers.

@chapmanb I'll do that.  Thanks for the suggestion.


Improve package compatibility with repology - U field missing inside APKIndex files

Problematic :
Repology cannot scan and expose the wolfi distribution package because the URL field inside the APKIndex yaml file is missing.
Summary :
Repology is the API used by Renovate and other tools to detect and automate version bump. After this improvement, we could have wolfi distribution integrated to repology and all apko for docker file could be scanned by Renovate to detect version bump of packages in a automated way.
Proposed changes :
Add the U field (URL field) inside APKIndex generated by melange. This U field could be populated through a new yaml declaration at package level
example
package:
  name: bash
  version: 5.2.21
  epoch: 5
  description: "GNU bourne again shell"
  url: "https://www.gnu.org/software/bash/bash.html"
  copyright:
    - license: GPL-3.0-or-later

Related issue : https://github.com/repology/repology-updater/issues/1339

I seems the url field is already available inside melange yaml file.
https://github.com/chainguard-dev/melange/blob/ad395d68d0c8e57ba2c1f4b261f7325ed2239b68/pkg/config/config.go#L87-L118
So the compatibility issue with repology is more on wolfi.dev/os  where this field is not set for each package


Typescript support

If anyone finds this one day and wants to help contribute, I'd appreciate any help. The current problem with Typescript support is that I need to use type module with Typescript, which I don't feel like working on currently.

I need to add the eslint typescript plugins as well.


[Aggregation] Make sure that aggregation cadence is not lost in distributed reduction.

This patch is attempting to make sure that queries containing a delta aggregation still returns a meaningful cadence.

@udoprog: Please review. We are not 100% sure we're fixing this in the right way.

@parmus to make this consistent it should probably be solved by propagating the current cadence similarly to how required tags are currently being propagated:
https://github.com/spotify/heroic/blob/master/heroic-component/src/main/java/com/spotify/heroic/aggregation/Chain.java
In effect add the 'last cadence' to the aggregation context and have the delta aggregation inherit that. Aggregations that change the cadence would do so appropriately by causing the context to be modified.
AggregationContext also appears to have bit-rotted a bit, defaultSize and defaultExtent should both be collapsed into a field called lastCadence or similarly. This is based of defaultExtent never being used which effectively makes defaultSize == lastCadence.

I forgot to mention that allowing the cadence to travel with the chain would permit us to not having to specify the sampling in every step when sending the query to the API. Something UI developers and users would benefit from.


fix multi schema broadcast null exception.

Pull Request Test Coverage Report for Build 7357

5 of 5 (100.0%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.009%) to 53.862%

Totals

Change from base Build 7356:
0.009%

Covered Lines:
8416

Relevant Lines:
15625

💛  - Coveralls


Clean up abbreviations

There are lots of instance in the documentation of both "PF" and "Patternfly" being used, as well as whether the 4 is included or not; this should probably be made more consistent.
There are also smaller things like "applauncher" or "app launcher" or should it really be named an application launcher? Also, "Nav" or "navigation"?

Awaiting update from Sarah regarding where to update documentation

One example is in https://pf4.patternfly.org/components/AppLauncher/examples/ where we have both "Applauncher" and "App launcher"
Another is here https://pf4.patternfly.org/components/Nav/examples/ where the menu and examples use "Nav" but the page is titled "Navigation"
I think maybe the PF vs Patternfly examples I noticed were in code comments.

:tada: This issue has been resolved in version 2.8.4 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[Snyk] Upgrade dayjs from 1.11.5 to 1.11.9

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to upgrade dayjs from 1.11.5 to 1.11.9.
As this is a private repository, Snyk-bot does not have access. Therefore, this PR has been created automatically, but appears to have been created by a real user.
:sparkles: Snyk has automatically assigned this pull request, set who gets assigned.
:information_source: Keep your dependencies up-to-date. This makes it easier to fix existing vulnerabilities and to more quickly identify and fix newly disclosed vulnerabilities when they affect your project.

The recommended version is 4 versions ahead of your current version.
The recommended version was released 2 months ago, on 2023-07-01.

Release notes

Package name: dayjs

1.11.9 - 2023-07-011.11.9 (2023-07-01)
Bug Fixes

Add null to min and max plugin return type (#2355) (62d9042)
check if null passed to objectSupport parser (#2175) (013968f)
dayjs.diff improve performance (#2244) (33c80e1)
dayjs(null) throws error, not return dayjs object as invalid date (#2334) (c79e2f5)
objectSupport plugin causes an error when null is passed to dayjs function (closes #2277) (#2342) (89bf31c)
Optimize format method (#2313) (1fe1b1d)
update Duration plugin add/subtract take into account days in month (#2337) (3b1060f)
update MinMax plugin 1. ignore the 'null' in args 2. return the only one arg (#2330) (3c2c6ee)

1.11.8 - 2023-06-021.11.8 (2023-06-02)
Bug Fixes

.format add padding to 'YYYY' (#2231) (00c223b)
Added .valueOf method to Duration class (#2226) (9b4fcfd)
timezone type mark date parameter as optional (#2222) (b87aa0e)
type file first parameter date is optional in isSame(), isBefore(), isAfter() (#2272) (4d56f3e)

1.11.7 - 2022-12-061.11.7 (2022-12-06)
Bug Fixes

Add locale (zh-tw) meridiem (#2149) (1e9ba76)
update fa locale (#2151) (1c26732)

1.11.6 - 2022-10-211.11.6 (2022-10-21)
Bug Fixes

add BigIntSupport plugin (#2087) (f6dce48)
Fix objectSupport collides with Duration plugin - issue #2027 (#2038) (c9370ea)

1.11.5 - 2022-08-121.11.5 (2022-08-12)
Bug Fixes

ordinal for nl not working (#2011) (c93c85e)
wrong ordinal for french locale (#2010) (dd192a7)

    from dayjs GitHub release notes

Commit messages

Package name: dayjs

5044cc8 chore(release): 1.11.9 [skip ci]
a4c3335 Merge pull request #2360 from iamkun/dev
1fe1b1d fix: Optimize format method (#2313)
62d9042 fix: Add null to min and max plugin return type (#2355)
3b1060f fix: update Duration plugin add/subtract take into account days in month (#2337)
3c2c6ee fix: update MinMax plugin 1. ignore the 'null' in args 2. return the only one arg (#2330)
013968f fix(): check if null passed to objectSupport parser (#2175)
c79e2f5 fix: dayjs(null) throws error, not return dayjs object as invalid date (#2334)
24e3038 chore: Added Hebrew version of Readme (#2340)
33c80e1 fix: dayjs.diff improve performance (#2244)
548b217 test: update test case
1a199d3 test: update test case
89bf31c fix: objectSupport plugin causes an error when null is passed to dayjs function (closes #2277) (#2342)
b3624de chore(release): 1.11.8 [skip ci]
880ac2f Merge pull request #2323 from iamkun/dev
daa1379 chore: update ci
a101def Merge pull request #2320 from iamkun/dev
851e774 chore: update action debug
8b6d0e5 Merge pull request #2319 from iamkun/dev
df49762 chore: update action node env
e2cffd3 Merge pull request #2318 from iamkun/dev
13d4ef9 chore: update action env
454cd09 Merge pull request #2317 from iamkun/dev
cfd4bc8 chore: update action build

Compare

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open upgrade PRs.
For more information:
🧐 View latest project report
👩💻 Set who automatically gets assigned
🛠 Adjust upgrade PR settings
🔕 Ignore this dependency or unsubscribe from future upgrade PRs

Fixed


Tracking Table: Which Functions Have Polars / Parallel Processing

Add a table to homepage with comparisons.

Polars Backend Tracking is happening here: https://github.com/business-science/pytimetk/issues/77
Polars Testing Tracking is happening here: https://github.com/business-science/pytimetk/issues/177
Polars Speed Comparison to Pandas is happening here: https://business-science.github.io/pytimetk/performance/01_speed_comparisons.html


email_password_sign_in_page exception

this error comes when I do a email password - signin or register
[it does log me in]
exception
E/flutter (22129): [ERROR:flutter/lib/ui/ui_dart_state.cc(157)] Unhandled Exception: Bad state: No element
[        ] E/flutter (22129): #0      List.last (dart:core-patch/growable_array.dart:227:5)
[        ] E/flutter (22129): #1      NavigatorState.pop (package:flutter/src/widgets/navigator.dart:2118:43)
[        ] E/flutter (22129): #2      _EmailPasswordSignInPageState._submit
(package:firebase_auth_demo_flutter/app/sign_in/email_password/email_password_sign_in_page.dart:80:20)
[        ] E/flutter (22129):
[        ] E/flutter (22129): #3      _InkResponseState._handleTap (package:flutter/src/material/ink_well.dart:706:14)
[        ] E/flutter (22129): #4      _InkResponseState.build. (package:flutter/src/material/ink_well.dart:789:36)
[        ] E/flutter (22129): #5      GestureRecognizer.invokeCallback (package:flutter/src/gestures/recognizer.dart:182:24)
[        ] E/flutter (22129): #6      TapGestureRecognizer.handleTapUp (package:flutter/src/gestures/tap.dart:486:11)
[        ] E/flutter (22129): #7      BaseTapGestureRecognizer._checkUp (package:flutter/src/gestures/tap.dart:264:5)
[        ] E/flutter (22129): #8      BaseTapGestureRecognizer.acceptGesture (package:flutter/src/gestures/tap.dart:236:7)
[        ] E/flutter (22129): #9      GestureArenaManager.sweep (package:flutter/src/gestures/arena.dart:156:27)
[        ] E/flutter (22129): #10     GestureBinding.handleEvent (package:flutter/src/gestures/binding.dart:222:20)
[        ] E/flutter (22129): #11     GestureBinding.dispatchEvent (package:flutter/src/gestures/binding.dart:198:22)
[        ] E/flutter (22129): #12     GestureBinding._handlePointerEvent (package:flutter/src/gestures/binding.dart:156:7)
[        ] E/flutter (22129): #13     GestureBinding._flushPointerEventQueue (package:flutter/src/gestures/binding.dart:102:7)
[        ] E/flutter (22129): #14     GestureBinding._handlePointerDataPacket (package:flutter/src/gestures/binding.dart:86:7)
[        ] E/flutter (22129): #15     _rootRunUnary (dart:async/zone.dart:1138:13)
[        ] E/flutter (22129): #16     _CustomZone.runUnary (dart:async/zone.dart:1031:19)
[        ] E/flutter (22129): #17     _CustomZone.runUnaryGuarded (dart:async/zone.dart:933:7)
[        ] E/flutter (22129): #18     _invoke1 (dart:ui/hooks.dart:273:10)
[        ] E/flutter (22129): #19     _dispatchPointerDataPacket (dart:ui/hooks.dart:182:5)
[        ] E/flutter (22129):
--
Thank you

I also face this issue when i try to login with email and password.

Is this happening with the latest master?

Yes @bizz84 this is happening in this project with latest master (v0.1.2).
E/flutter ( 7886): [ERROR:flutter/lib/ui/ui_dart_state.cc(157)] Unhandled Exception: Bad state: No element
E/flutter ( 7886): #0      List.last (dart:core-patch/growable_array.dart:227:5)
E/flutter ( 7886): #1      NavigatorState.pop (package:flutter/src/widgets/navigator.dart:2118:43)
E/flutter ( 7886): #2      _EmailPasswordSignInPageState._submit (package:firebase_auth_demo_flutter/app/sign_in/email_password/email_password_sign_in_page.dart:80:20)
E/flutter ( 7886): <asynchronous suspension>
E/flutter ( 7886): #3      _InkResponseState._handleTap (package:flutter/src/material/ink_well.dart:706:14)
E/flutter ( 7886): #4      _InkResponseState.build.<anonymous closure> (package:flutter/src/material/ink_well.dart:789:36)
E/flutter ( 7886): #5      GestureRecognizer.invokeCallback (package:flutter/src/gestures/recognizer.dart:182:24)
E/flutter ( 7886): #6      TapGestureRecognizer.handleTapUp (package:flutter/src/gestures/tap.dart:486:11)
E/flutter ( 7886): #7      BaseTapGestureRecognizer._checkUp (package:flutter/src/gestures/tap.dart:264:5)
E/flutter ( 7886): #8      BaseTapGestureRecognizer.handlePrimaryPointer (package:flutter/src/gestures/tap.dart:199:7)
E/flutter ( 7886): #9      PrimaryPointerGestureRecognizer.handleEvent (package:flutter/src/gestures/recognizer.dart:467:9)
E/flutter ( 7886): #10     PointerRouter._dispatch (package:flutter/src/gestures/pointer_router.dart:76:12)
E/flutter ( 7886): #11     PointerRouter._dispatchEventToRoutes.<anonymous closure> (package:flutter/src/gestures/pointer_router.dart:117:9)
E/flutter ( 7886): #12     _LinkedHashMapMixin.forEach (dart:collection-patch/compact_hash.dart:379:8)
E/flutter ( 7886): #13     PointerRouter._dispatchEventToRoutes (package:flutter/src/gestures/pointer_router.dart:115:18)
E/flutter ( 7886): #14     PointerRouter.route (package:flutter/src/gestures/pointer_router.dart:101:7)
E/flutter ( 7886): #15     GestureBinding.handleEvent (package:flutter/src/gestures/binding.dart:218:19)
E/flutter ( 7886): #16     GestureBinding.dispatchEvent (package:flutter/src/gestures/binding.dart:198:22)
E/flutter ( 7886): #17     GestureBinding._handlePointerEvent (package:flutter/src/gestures/binding.dart:156:7)
E/flutter ( 7886): #18     GestureBinding._flushPointerEventQueue (package:flutter/src/gestures/binding.dart:102:7)
E/flutter ( 7886): #19     GestureBinding._handlePointerDataPacket (package:flutter/src/gestures/binding.dart:86:7)
E/flutter ( 7886): #20     _rootRunUnary (dart:async/zone.dart:1138:13)
E/flutter ( 7886): #21     _CustomZone.runUnary (dart:async/zone.dart:1031:19)
E/flutter ( 7886): #22     _CustomZone.runUnaryGuarded (dart:async/zone.dart:933:7)
E/flutter ( 7886): #23     _invoke1 (dart:ui/hooks.dart:273:10)
E/flutter ( 7886): #24     _dispatchPointerDataPacket (dart:ui/hooks.dart:182:5)

Hi, does anyone know a solution to this bug?

Hi, does anyone know a solution to this bug?

Ok so it appears that this only happens in the emulator. When testing the app on my physical device (iphone) it does not have the bug.


Sharing a library between two apps

Versions
Go: go version go1.8.3 linux/amd64
Dep: v0.1.0-286-g0a63c47
Expected behavior
I have the following directory structure. I'm expecting the shared directory to have files that will be linked to both api-server and dbcli (some files regarding schema and DB models). When I try to run deps ensure I get the error:
Commands executed
cd src/acme.in/api-server && ../../../bin/dep ensure
cd src/acme.in/dbcli && ../../../bin/dep ensure

Error
panic: shouldn't be possible unable to deduce repository and source type for "acme.in/shared/lib/config": unable to read metadata: go-import metadata not found

Here's my project directory structure. api-server is a web server (its own project) and dbcli is a CLI for managing the DB records, also it's own project. I want the shared folder to be importable in both apps.
src/acme.in/
├── api-server
│   ├── api-server.go
│   ├── Gopkg.lock
│   ├── Gopkg.toml
│   ├── lib
│   │   └── rest-utils
│   │       └── rest.go
│   └── routes
│       └── routes.go
├── dbcli
│   ├── dbcli.go
│   ├── Gopkg.lock
│   ├── Gopkg.toml
│   └── lib
│       ├── config
│       │   └── config.go
│       ├── persistence
│       │   ├── mongo.go
│       │   └── persistence.go
│       └── utils
│           └── utils.go
└── shared
    ├── Gopkg.lock
    ├── Gopkg.toml
    └── lib
        ├── config
        │   └── config.go
        └── persistence
            ├── mongo.go
            └── persistence.go

I'm including files in routes/routes.go like so:
import (
	"acme.in/shared/lib/config"
	"acme.in/shared/lib/persistence"
)

And in dbcli/dbcli.go like so:
import (
	"acme.in/shared/lib/config"
	"acme.in/shared/lib/persistence"
)

What am I missing here? How can I do code reuse across apps easily

This is related to #174.

panic: shouldn't be possible unable to deduce repository and source type for "acme.in/shared/lib/config": unable to read metadata: go-import metadata not found

This error means thathttps://acme.in/shared/lib/config?go-get=1 or (http://) didn't return valid go-get meta data for dep to be able to deduce the repository and type.

BTW, I'm assuming that acme.in/shared is available on a remote repository. :grin:
If it's not then that's another issue.

It is a local folder, what are the requirements for naming local folders Vs remote repos?

Also 'go get -d acme.in/api-server' works as expected

@elefthei go get doesn't download the package if the folder exists in GOPATH/src. go get -u should be failing if you try it.
At the moment, dep doesn't fetch dependencies from GOPATH. See @sdboyer comment in #792 and the gist for more details on the multi-project workflow.
@jonmclachlan The panic should be handled and an appropriate error message should be displayed, but we don't support local repositories, at least atm. 😞


python310Packages.marshmallow-dataclass: 8.5.12 -> 8.5.13

Diff: https://github.com/lovasoa/marshmallow_dataclass/compare/refs/tags/v8.5.12...v8.5.13
Changelog: https://github.com/lovasoa/marshmallow_dataclass/blob/v8.5.13/CHANGELOG.md
Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 227684 run on x86_64-linux 1

17 packages built:

home-assistant-component-tests.renault
python310Packages.cairo-lang
python310Packages.cairo-lang.dist
python310Packages.lektor
python310Packages.lektor.dist
python310Packages.marshmallow-dataclass
python310Packages.marshmallow-dataclass.dist
python310Packages.renault-api
python310Packages.renault-api.dist
python311Packages.cairo-lang
python311Packages.cairo-lang.dist
python311Packages.lektor
python311Packages.lektor.dist
python311Packages.marshmallow-dataclass
python311Packages.marshmallow-dataclass.dist
python311Packages.renault-api
python311Packages.renault-api.dist


Clear thread local values on UTF8TaxonomyWriterCache.close()

UTF8TaxonomyWriterCache keeps a ThreadLocal BytesRefBuilder, the bytes for which, are not removed on thread close. This leads to memory leaks.
The change uses CloseableThreadLocal instead, and closes out the object on cache.close()
Addresses #12000

Thanks, it looks better to use CloseableThreadLocal than ThreadLocal... but I don't understand what this "cache" is doing and why it actually documents that it never frees memory. Especially as a default!
As a followup can we open an issue to move away from Threadlocal storage here by default? I don't understand why we have this "cache" with this description:
/** A "cache" that never frees memory, and stores labels in a BytesRefHash (utf-8 encoding). */

Also, I'm trying to remove ThreadLocal (and CloseableThreadLocal) usages elsewhere in the codebase (e.g. #11998). These threadlocals are problematic in java apps that have many threads (or high thread churn rate), just as CloseableThreadLocal documents and attempts to workaround.
At a glance, seems to me that using a bounded LRU cache would be much more reasonable...

Also (summoning my inner @uschindler), curious what the perf impact is if we simply remove these caches completely. Maybe they are not relevant to the performance anymore due to faceting changes (e.g. storing labels in docvalues rather than payloads) ? Maybe they are even hurting performance?

Please remove the cache. Totally useless. It hurts more because the escape analysis can't work.
When tiered compilation stepped in, this is just a useless extra
My rule: Never ever cache object instances to prevent object creation and help GC. This was needed in in Java 6 and java 7, but not anymore. It has opposite problem: it actually creates those objects on heap and stresses GC.

Yeah, I'm highly suspicious of this cache. I dug into this and found that previously stored fields (!) were used for this lookup. So no surprise there was a cache around it, since this is a slow approach!!!
But @gautamworah96 fixed this, and these days BinaryDocValues are used instead: see #10490
IMO there is no sense in caching BinaryDocValues lookups into the heap, it is already mmap'd and the operating system will do this.

... but I don't understand what this "cache" is doing and why it actually documents that it never frees memory.

My understanding is that TaxonomyWriterCache caches ordinals for all categories created in the index so far, so that categories use the same ordinals when facet labels are added.
It seems that we need such a cache to get ordinals for newly added categories from documents that are still pending flush.
From TaxonomyWriterCache#put() docstring:
   * <p>The reason why the caller needs to know if part of the cache was cleared is that in that
   * case it will have to commit its on-disk index (so that all the latest category additions can be
   * searched on disk, if we can't rely on the cache to contain them).

However, with faster BinaryDocValue fields, maybe we don't need the "never evicting UTF8TaxonomyWriterCache" anymore. And we could make LruTaxonomyWriterCache as the default?
I can close this PR and make that change if it makes sense. Or we can merge this change and take it up in a separate issue. Are there faceting specific benchmarks that can help validate the change?

OK, crazy that having no cache at all could cause unnecessary amounts of flush/reopens. We want at least a small one.

I can close this PR and make that change if it makes sense. Or we can merge this change and take it up in a separate issue. Are there faceting specific benchmarks that can help validate the change?

+1 to make the LRU cache the default. And please, lets also remove this ThreadLocal-based cache completely in main branch, and deprecate in 9.5

Created a separate PR - #12092 to remove support for UTF8TaxonomyWriterCache from main. Will close this PR.

PR - https://github.com/apache/lucene/pull/12093 to deprecate UTF8TaxonomyWriterCache in 9.x


fixes#6143 changed the error message

Fixes #6143
changed the error message displayed when local state is empty
@nazmulidris

@Salonikansal2 while we still have to review this change. I'm adding the HACKTOBERFEST-ACCEPTED label so the contribution is counted by hacktoberfest bots before the end of October. Thank you for taking time to contribute 🎉

Thanks for your efforts @mrinalwadhwa

gentle reminder to review changes
@adrianbenavides
@nazmulidris

Gentle reminder @mrinalwadhwa , @nazmulidris , @adrianbenavides  . Sorry for such hurry in behavior but actually I participated in hacksquad competition due to which I need them merged as soon as possible

@metaclips

Sorry , for the late reply ,was busy in college.
Just summited my new pr #6808 by making the changes you suggested
@adrianbenavides
Thanks for your time

Closed in favor of: https://github.com/build-trust/ockam/pull/6808


Containerlab optimization, e.g., memory deduplication

systemd >= 254 should support MemoryKSM=true, if kernel >= 6.4, but this currently only works for compiled program and not together with systemd

I moved this issue to the backlog, though PR_SET_MEMORY_MERGE is supported in systemd >= 254, it did not work neither in recent debian (testing & unstable), ubuntu or fedora. I was able to build a binary that sets PR_SET_MEMORY_MERGE and also successfully used LD_PRELOAD and identical mem pages were deduplicated correctly. However, systemd throws errors as soon as MemoryKSM: true is set. To me it seams like the systemd implementation misses setting several additional parameters for PR_SET_MEMORY_MERGE, as only one param is used.


No presidential bios

The presidents do not have bios, because they are not in the congressional guide. We should source short bios somewhere and scrape them (excluding, perhaps, the current president?) to solve this.

After a writing marathon today, this is done, including 45. And it's been integrated into the workflow build.
I wrote in the bioguide style and listed the same stuff they did (birth date, birth place, lower education schools, universities, degrees earned, election results, any notable awards, place of retirement, date and manner of death, place of internment).


Facing problem when tries to delete a record with Elasticquent

Hello, I made a delete call:
static::deleted(function ($user) { 			$user->deleteFromIndex(); 		});
gets error when I call eloquent delete
BadMethodCallException in Builder.php line 2161: Call to undefined method Illuminate\Database\Query\Builder::deleteFromIndex()

Fixed my using:
static::deleted(function ($user) { 			$user->getElasticSearchClient()->delete($user->getBasicEsParams()); 		});

deleteFromIndex was not a function that used to be here (until today!)

Thanks @nagibmahfuj, was having this issue and your solution seems to work.


Regular crashing of Windows host nodes due to BSOD / BugCheck

Is this a request for help?: Yes

Is this an ISSUE or FEATURE REQUEST? (choose one): Issue

What version of acs-engine?: v0.20.9

Orchestrator and version (e.g. Kubernetes, DC/OS, Swarm)
Kubernetes 1.11.2
What happened:
It seems that Windows is restarting unexpectedly, especially when there is a lot of churn in running containers. I am unsure if this is the VM image being used by acs-engine or a general windows bug. I have captured memory.dmp's from machines that have rebooted unexpectedly and opened them to determine the crash

Probably caused by : vmswitch.sys ( vmswitch!VmsExtMpSendNetBufferLists+886 )

Followup:     MachineOwner
---------

nt!KeBugCheckEx:
fffff801`4ee4eca0 48894c2408      mov     qword ptr [rsp+8],rcx ss:0018:ffffab01`c0589db0=000000000000007f

The windows server version I am using is 1803 and I have stored 2 memory dumps with the same BugCheck.
What you expected to happen:
Not BSOD and reboot unexpectedly.
How to reproduce it (as minimally and precisely as possible):
This happens irregularly. I can supply memory.dmp files however I am unsure the best place to do so.
Anything else we need to know:
As it stands, I don't feel Windows containers are production grade, which is a shame. Either that or the base image of the windows containers on the version I am using is buggy and upgrading will fix, if that is the case that would be awesome.

@PatrickLang

just an update, I opened a support request to Azure support and sent the crash dumps to them. They have acknowledged and engineers are investigating internally. I'm not sure if case numbers are sensitive but can share to anyone if you want to see it.

I have had the exact same issue today. Azure support looked into it for me as the VM wouldn't come back online. Basically after the crash I ended up with two NIC's with the same mac address but different IPs. Once sorted out the node is back however pods no longer work on it. I am getting the below:
Failed create pod sandbox: rpc error: code = Unknown desc = [failed to set up sandbox container "a73aded9a1542c302821a11f633cb409bbdc5bf63cd1234460ba1a5dd1061b2e" network for pod "rifleman-pdf-generation-uat1-ddfbd8b6f-mxqn5": NetworkPlugin cni failed to set up pod "rifleman-pdf-generation-uat1-ddfbd8b6f-mxqn5_default" network: netplugin failed but error parsing its diagnostic message "": unexpected end of JSON input, failed to clean up sandbox container "a73aded9a1542c302821a11f633cb409bbdc5bf63cd1234460ba1a5dd1061b2e" network for pod "rifleman-pdf-generation-uat1-ddfbd8b6f-mxqn5": NetworkPlugin cni failed to teardown pod "rifleman-pdf-generation-uat1-ddfbd8b6f-mxqn5_default" network: netplugin failed but error parsing its diagnostic message "": unexpected end of JSON input]

This seems related to https://github.com/Azure/azure-container-networking/issues/242 (also https://github.com/Azure/azure-container-networking/issues/232 for the error logging problem). @ocdi do you see anything similar in the kubelet logs?

@tonyschollum this is not related to windows crashing with BSOD. Can you please open a separate issue for this and attach c:\k\azure-vnet.log.* files

@CecileRobertMichon annoyingly I killed the nodes that were affected. I have a single node left running from that cluster and am trying to trigger a BSOD by killing containers and having it restart them (that did seem to be related / trigger it). The windows containers are ASP.NET apps on full framework so I would be surprised if there was any particular traffic causing it, but it does seem with less network traffic going through there have been less crashes. I am too nervous to put production traffic on the cluster again unless we find an actual cause or definitive resolution.
I'm hoping the team looking at my support request find something looking at the crash dumps. :-)

I heard back from support this morning. Apparently the fix will be in the upcoming windows release (ie the one after 1809), and the team are looking at back-porting the fix to earlier Windows releases. How and when that will end up on my cluster, I have no idea. For now I am running static windows servers and not using containers. I suspect we could close this now.


feat(inline-select): Add inline-select component

Add Inline Select
Added
html and scss for inline-select
Note: width of the select can be set so that there is not as much whitespace to the left of the arrow, but by default, it will fit the width of its container (bx--form-item)

Can you take a look at the original select as well? It's hidden until you focus on it with tab key

@hellobrian I think that's just due to the demo-container having a min-height: 2000px on it, and display: flex; justify-content: space-between. If you remove those styles it should render correctly and not scroll down the screen

👍 ✅


Data lost during synchronization

Hello,
This week we had to work as a team, and we decided to use Qfield Cloud.
When the data was done recording, I synchronized the data.
On the new project, I can no longer see the recorded data. Everything from yesterday disappeared.
Maybe I did a false manipulation, I'm not really sure.
Do you have any idea where it could come from ?
On QGIS, the synchronizing widow doesn't show the layer that we need: "Centroid_Grid_100x100".
The project name is: Siargao_MangroveHeath_UST
User: jeb
Thanks.

@Ipickedausername when you say "everything diappeared", do you still see your modifications in the the web interface, in the "Project -> changes" page?

I see a large amount of lines in "Changes".
The last one was problematic. The pictures and info of the different layers have been synchronized, except features from the layer "centroid_grid_100x100".
It's been an update of a list of centroids.
I just remembered that i tik the option to manually check the conflicts. Maybe it comes from it.

I can't resolve the conflicts. We I change something I have "Server Error (500)"

Hi,
Personally, I stopped using Qfield cloud. I had many issues. I still have to make new tests about this issue.


Shanghai Research Center for Brain Science & Brain-Inspired Intelligence

Shanghai Research Center for Brain Science & Brain-Inspired Intelligence
Official website: http://www.bsbii.cn/

@puppy2333 I am sorry, I have to close this request without merging, because this organization is a pure research institution, not a university, not a college. If employees of this Institute who are not students, nor teachers, need a license for JetBrains tools, they are eligible for an academic discount for the tools. They can get the discount by email to<EMAIL_ADDRESS>If there are students who make their studies in the Institute, please ask them to apply for free educational license foe JetBrains tools with official document ("OFFICIAL DOCUMENT" tab at https://www.jetbrains.com/shop/eform/students).


[FLINK-2658] fieldsGrouping for multiple output streams fails

added SplitStreamTypeKeySelector and JUnit tests

Can I merge this?

This does not affect any code outside the storm compatibility layer, so I think if you judge it as solid, it is probably fine...


[Enhancement] (nereids)implement DropStoragePolicyCommand  in nereids

Search before asking

[X] I had searched in the issues and found no similar issues.

Description
DropStoragePolicyCommand is supported in the legacy planner but needs implementation in Nereids. Use the syntax definition in DorisParser.g4, and refer to DropPolicyStmt.java in the legacy planner.
unsupportedDropStatement : DROP STORAGE POLICY (IF EXISTS)? name=identifier
Solution
No response
Are you willing to submit PR?

[ ] Yes I am willing to submit a PR!

Code of Conduct

[X] I agree to follow this project's Code of Conduct

@LiBinfeng-01 @starocean999 I will pick this up, thanks


hubble: Stop building 32-bit binaries

Similar to cilium-cli ^1, it's time to move on and drop 32-bit binaries from release artifacts.

hubble-cli workflow run ✅ https://github.com/cilium/cilium/actions/runs/11827181145/job/32954787410?pr=35974

/test

added backport-1.16 label since we cut hubble-cli releases from the latest stable branch 🚀🙏


Unable to save weights and reload them in another model via JSON

Please make sure that the boxes below are checked before you submit your issue. Thank you!

[x] Check that you are up-to-date with the master branch of Keras. You can update with:
pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps
[x] If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps
[x] Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

Steps to reproduce: add following lines to end of keras/examples/babi_memnn.py to save the JSON and model weights to files. Reason I am using this example is because I have similar network (with Merge layers) and this is self contained (pulls its own data, etc) so its easy to run.
answer.save_weights("/tmp/babi_memnn_copy.h5")
with open("/tmp/babi_memnn_copy.json", "wb") as fjson:
    fjson.write(answer.to_json())

Run the code, and once it completes attempt to run the following code to retrieve the model.
from keras.models import model_from_json

with open("/tmp/babi_memnn_copy.json", "r") as fjson:
    model = model_from_json(fjson.read())
model.load_weights("/tmp/babi_memnn_copy.h5")

Gives following exception:
Using Theano backend.
Traceback (most recent call last):
  File "reload_babi_memnn_copy.py", line 5, in <module>
    model.load_weights("/tmp/babi_memnn_copy.h5")
  File "/Users/palsujit/anaconda/lib/python2.7/site-packages/keras/engine/topology.py", line 2500, in load_weights
    self.load_weights_from_hdf5_group(f)
  File "/Users/palsujit/anaconda/lib/python2.7/site-packages/keras/engine/topology.py", line 2552, in load_weights_from_hdf5_group
    str(len(flattened_layers)) + ' layers.')
Exception: You are trying to load a weight file containing 5 layers into a model with 6 layers.

Also tried with YAML files, same error message.
Since the reason one would train a model is to be able to use it in another process to do predictions, it seems an important functionality.
Strangely enough, when I try a slightly modified version of the snippet in Issue 2281 as follows:
from keras.models import Sequential, model_from_json
from keras.layers import *

model = Sequential()
model.add(Convolution2D(32, 5, 5,
                        border_mode='valid',
                        input_shape=(3, 216, 384)))
model.add(Activation('relu'))
model.add(Convolution2D(32, 5, 5))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Convolution2D(64, 3, 3, border_mode='valid'))
model.add(Activation('relu'))
model.add(Convolution2D(64, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(8))
model.add(Activation('relu'))
model.add(Dropout(0.))

model.add(Dense(9))
model.add(Activation('linear'))

model.save_weights('test.h5')
config = model.to_json()
open("test.json", "wb").write(config)

followed by an attempt to reconstruct it from the h5 and json...
from keras.models import Sequential, model_from_json
from keras.layers import *

config = open("test.json", "rb").read()
model = model_from_json(config)
model.load_weights('test.h5')

This succeeds without any exception being thrown.

Closed by mistake, sorry, reopening.

Man, just try
model.save(file)
m = load_model(file)

for save or load model. It save compiled model with weights and you don't need additional working with weights.

Thanks for the answer @vrodionovpro. Unfortunately it didn't help. I get identical behavior with this as I reported originally.
I replaced the snippet in babi_memnn.py with the following:
answer.save("/tmp/babi_memnn_copy.model")

and now my corresponding code to load this back from disk looks like this:
from keras.models import load_model

model = load_model("/tmp/babi_memnn_copy.model")
print(model)

Building the model works without complains as before, but when I try to get the model back, I get the same exception.
Using Theano backend.
Traceback (most recent call last):
  File "reload_babi_memnn_copy.py", line 16, in <module>
    model = load_model("/tmp/babi_memnn_copy.model")
  File "/Users/palsujit/anaconda/lib/python2.7/site-packages/keras/models.py", line 131, in load_model
    model.load_weights_from_hdf5_group(f['model_weights'])
  File "/Users/palsujit/anaconda/lib/python2.7/site-packages/keras/engine/topology.py", line 2552, in load_weights_from_hdf5_group
    str(len(flattened_layers)) + ' layers.')
Exception: You are trying to load a weight file containing 5 layers into a model with 6 layers.

I did a pip install as required by the issue reporting pre-requisites and my Keras version is 1.1.0 and theano version is 0.9.0dev2.dev-dab0b393a743331c179916680a2fd5ab45759d81.
Thanks.

I rolled back to the following stable versions according to this post: Theano 0.8.5 and Keras 1.0.5 but still no luck. However, I did a little bit of debugging that might be useful, so putting it here:
I tried matching model.flattened_layers and h5py.File.attrs["layer_names"] since that was what the error message was looking at:
layer_names      model.flattened_layers                                                      variable in code
------------   --------------------                                                    --------------
'embedding_1'  <keras.layers.embeddings.Embedding at 0x117bea150>  input_encoder_m
'dropout_1'       <keras.layers.core.Dropout at 0x117e3f590>                   input_encoder_m
'embedding_2'  <keras.layers.embeddings.Embedding at 0x117e76050>  question_encoder
'dropout_2'       <keras.layers.core.Dropout at 0x117ef5450>                    question_encoder
'embedding_3'  <keras.layers.embeddings.Embedding at 0x117f1df10>   input_encoder_c
'dropout_3'       <keras.layers.core.Dropout at 0x117f315d0>                   input_encoder_c
'permute_1'      <keras.layers.core.Permute at 0x117fb8090>                   response
                        <keras.layers.embeddings.Embedding at 0x117fb8210>
                        <keras.layers.core.Dropout at 0x117fc35d0>
'lstm_1'             <keras.layers.recurrent.LSTM at 0x117fc3f90>                 answer
'dropout_4'       <keras.layers.core.Dropout at 0x118250350>                  answer
'dense_1'          <keras.layers.core.Dense at 0x118249890>                      answer
'activation_1'    <keras.layers.core.Activation at 0x118250510>                answer

There are 3 merges in this model, match = (input_encoder_m dot question_encoder), response = (match plus input_encoder_c) and answer = (response concat question_encoder), which I am guessing flattened_layers does not list.
However, there is one embedding+dropout layer that are listed in flattened_layers but not in layer_names. The model.summary() output shows that these correspond to repeats of embedding_2 and dropout_2, which is the RHS of the 3rd Merge operation.
So perhaps we need to replace the check with whatever the model.summary() is doing to get these layers?

I did some more google searches following the above, and came across Issue 1508, where the OP (@ymcui) explains the (similar) problem better, and finally concludes that he can use the Graph model instead. That prompted me to look at the Graph model (which is deprecated and replaced by the functional API). I was able to successfully save and load models using the functional API.
I have put up my modified code for examples/babi_memnn.py using the functional API in the gist here in case it is useful to someone.
So I have a workaround, please advise if I should close the bug or if you would prefer to fix it in the Sequential model as well. OTOH, if users are supposed to use the Functional API instead of the Sequential API once inputs are reused, then perhaps this should be reflected in the documentation.

I have the same problem with an adapted InceptionV3 model based on the application-classes in keras 1.2.0 with up-to-date theano:
from keras.layers import Dense, Input, Flatten, GlobalAveragePooling2D
from keras.models import Model
from keras.applications.inception_v3 import InceptionV3
from keras.layers.core import Dense, Activation, Dropout, Flatten

base_model = InceptionV3(weights='imagenet', include_top=False, input_shape=(3, 256, 200))

top = GlobalAveragePooling2D(name='flatten')(base_model.output)
top = Dense(1024, activation='relu')(top)
top = Dropout(0.2)(top)
top = Dense(1024, activation='relu')(top)
top = Dropout(0.5)(top)
top = Dense(output_units, activation='sigmoid', name='predictions')(top)

model = Model(base_model.input, top, name='vgg16')

for layer in base_model.layers:
	layer.trainable = False

I saved the models in both possible ways:
model.save('model.h5')

with open('model.json', "w") as json_file:
	json_file.write(model.to_json())

model.save_weights('model_weights.h5')

and loaded them accordingly:
# full model
model = load_model('model.h5')

# partial loading
json_file = open('model.json', 'r')
loaded_model_json = json_file.read()
json_file.close()
loaded_model = model_from_json(loaded_model_json)

loaded_model.load_weights("model_weights.h5")

Either way I get the same error message:
ValueError: You are trying to load a weight file containing 191 layers into a model with 3 layers.

I just tested it with a smaller network consisting of 3 convolutional and one fully connected layer and it worked. The problem seems to be related to the applications module. Maybe it would be better to move this into a new issue.

@slychief I came across the same problem as you.It says You are trying to load a weight file containing 16 layers into a model with 3 layers. Have you found the solution?

No, but it seems to be a problem with the applications module. I can store and load my own models, but not those using pre-trained imagenet weights.
Von: Nicholas-tien<EMAIL_ADDRESS>Gesendet: Montag, 6. März 2017 16:58
An: fchollet/keras<EMAIL_ADDRESS>Cc: Schindler Alexander<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Betreff: Re: [fchollet/keras] Unable to save weights and reload them in another model via JSON (#3927)
@slychiefhttps://github.com/slychief I came across the same problem as you.It says You are trying to load a weight file containing 16 layers into a model with 3 layers. Have you found the solution?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/fchollet/keras/issues/3927#issuecomment-284438605, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAyxy3Z8tpE5wDj-8KZLpSvJOvEDVTUDks5rjCzFgaJpZM4KLewP.

Hope someone can solve it.


os/exec: StdoutPipe example fails when run on golang.org

On the official doc, the example under func (*Cmd) StdoutPipe fails with error: 2009/11/10 23:00:00 exec: "echo": executable file not found in $PATH when run.

What operating system are you using?

I'm in Fedora 22 but to be clear this is the embedded example in the web page that fails, not me running anything locally.

Ah, I see.  Yes, none of the os/exec examples will actually work in the limited environment of the Go playground.  We should try to document that somehow.

I have a few ideas to this

We can document this for each example ie

// Note: This example when run in the playground
// will fail because the playground is sandboxed/limited
// so shell utility lookups will fail.

2a) Easily detect when we are running on the playground like this
import "runtime"

func execPathsUnavailable() bool {
     // Shell lookups and utilities are
     // unavailable on the playground running nacl
     return runtime.GOOS == "nacl"
}

2b) A little more complex:

Use the execPathsUnavailable() when extracting each example in exec/ detect and inject that comment before the body of each example.

Please let me know what you think would be a better option. Thanks.

os/exec is unavailable in nacl, we could just make
the error from os/exec better on nacl.

Could the 'run' be removed or disabled?

@stevekuznetsov, yes, that's probably the right answer.
If we don't want to invent new annotation syntax, we could even just have a blacklist of code samples in hard-coded in cmd/godoc. If an example is on the blacklist then it'd get to Run button.

We just need to blacklist the os/exec package to start with.

If you're running your own godoc then the example runs fine, right?


wip

A general idea how to handle exceptions and retries.

Not sure if I'm 100% liking this approach.. its a bit unreadable at some points.

I might also drag the retry out of Frisp into it's own package, so we can reuse it in our other tools..


tbb: Fix build for aarch64-darwin

Description of changes
Fix build for aarch64-darwin of package tbb by applying patch from upstream.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[x] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Builds for me,  macos 12.4 aarch64.
nixpkgs-review happy, macos 10.15


Pre-Process Doesn't Handle Newlines

Description
The current method for parsing pre-processed data doesn't handle newlines in process stdout, resulting in everything being on the same line.
Reproducing
Taking the example in the pre-processing example slide:
~~~graph-easy --as=boxart
[ A ] - to -> [ B ]
~~~

Produces:
┌───┐  to   ┌───┐ │ A │ ────> │ B │ └───┘       └───┘

Explanation
The output of the command being run is just added to the Block representing the whole slide (internal/process/process.go:68), which then all gets parsed by the markdown renderer (the newlines remain intact until internal/model/model.go:183). As a markdown newline is two newlines, and processes just output single newlines, the renderer interprets the process output with no newlines.
Temporary Solution
I didn't open a PR because my solution is quite hacky and probably breaks some cases, but you can wrap the process output in a markdown code block so the renderer interprets it as plain text. There's almost certainly a more elegant way to do it :'D
See here: https://github.com/Thomas-Malcolm/slides/commit/15990c5772ede761bc397120ae1dd03d819533a7

If new lines are important to preserve (without wrapping) I surround the preprocessing with markdown quotes in the slides directly so that this can be decided on a case by case basis.

```
~~~grapheasy --boxart
[ A ] -> [ B ]
~~~
```

Does this work for you?

Ah yeh I see how that should work now, interestingly it doesn't though, I still get the single-line output - time for system info!
$ uname -a
Linux <hostname> 5.10.0-10-amd64 #1 SMP Debian 5.10.84-1 (2021-12-08) x86_64 GNU/Linux
$ echo $SHELL
/bin/bash
$ go version
go version go1.17.6 linux/amd64

Interesting, can you try running these slides https://github.com/maaslalani/slides/blob/main/examples/preprocess.md
(Double check that the file is executable chmod +x preprocess.md)
There may be a bug if that still doesn't work. I can fix it if thats the case.

Can confirm I get the same error on those slides

Thanks so much for confirming, I'll take a look at what's going on.
Also thanks so much for using slides and reporting this issue ❤

Strange that it works for me 🤔

Strange, I just got back to my laptop, tried it on the same one and it... works now? Same slides binary in ~/go/bin and same slide example as before. There shouldn't be anything that causes non-deterministic behaviour right?
I'll be using it over the next day or two a fair bit, suppose if it happens again I'll drop another comment in here

The only thing I can think of is if the result of the pre-process was cached / using an old file of the result. But, I don't think that should be the case.
Glad it works now, do let me know if this happens again! Thanks so much for double checking as well 🙂

I can't get this to work either

I installed slides using snap, my terminal is Kitty using ZSH with powerlevel10k theme.

@Thomas-Malcolm Hey! Guessing this wasn't reproducible and just a weird blip, do re-open this issue if you encounter this again 😄


Print out OpenGL info on start.

Quick hack to display the OpenGL version, GFX and so on ....
$ ./konstructs
OpenGL vendor: NVIDIA Corporation
OpenGL version: 3.3.0 NVIDIA 340.96
OpenGL renderer: GeForce GTX 780/PCIe/SSE2
OpenGL extensions: (null)

... I'm not sure that this will work if nanogui fails to initialize the opengl 3.3 context. A better solution is probably to add this when we open the opengl context inside nanogui. Not sure that I like the idea to continue adding/changing code in our fork.

No point to merge this, I think.


Update Shields.io documentation.

Following from https://github.com/gratipay/gratipay.com/issues/4328.

Merging following http://inside.gratipay.com/howto/develop-software:

You can self-merge PRs that only change documentation, though you should still use PRs rather than committing directly to master. More docs!

Ah, good to have a doc for this. 😃


services of the default network of external IPAM get IP address assigned by internal Arktos

What happened:
The default network template specifies the service.ipam is External; DNS servcei of this network network still got ip address assigned by kuba-apiserver
What you expected to happen:
DNS serfice should have no clusterIP
How to reproduce it (as minimally and precisely as possible):

prepare a default-network template, having content of

{
    "metadata": {
        "name": "default",
        "finalizers": ["arktos.futurewei.com/network"]
    },
    "spec": {
        "type": "vpc",
        "service": {
             "ipam": "External"
        }
    }
}

start single node arktos lab by

export ARKTOS_NETWORK_TEMPLATE=<path-to-teh-default-network-template-file>
./hack/arktos-up.sh

prepare the local config file used by arktos-network-controller, with follwoing content

apiVersion: v1
clusters:
- cluster:
    server: http://<IP_ADDRESS>:8080
  name: local
contexts:
- context:
    cluster: local
    user: ""
  name: local-ctx
current-context: local-ctx
kind: Config
preferences: {}
users: []

build arktos-network-controller and start it by following command

./_output/local/bin/linux/amd64/arktos-network-controller --kubeconfig=<path-to-the-above-config-file> --kube-apiserver-ip=<ip-addr-of-this-host>

then, looking at the DNS service of the default network

$ ./cluster/kubectl.sh get svc kubernetes-default
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)
kubernetes-default   ClusterIP   <IP_ADDRESS>   <none>        443/TCP

Anything else we need to know?:
For custom networks, instead of the default one, their DNS services won't have ip address assigned, just as expected.
Environment:

Arktos version (use kubectl version): GitCommit:"94992457ec5098ae5af5b63da2eb20dae3c44803"
Cloud provider or hardware configuration: local dev machine
OS (e.g: cat /etc/os-release): ubuntu 18.04 LTS
Kernel (e.g. uname -a): 4.15.0-112-generic
Install tools: arktos-up.sh
Network plugin and version (if this is a network-related bug): flannel
Others:

external ipam not priority. pounded


Conflict terminator was fixed and now can work correctly when connecting to the PGBouncer.

A psycopg's function get_backend_pid returns a wrong pid when you are connecting to the
PGBouncer instead of the real Postgresql.
That is why it is better to use additional request to call pg_backend_pid().
More details in the https://github.com/psycopg/psycopg2/issues/955

Hello, Alexander.
We should probably use a bit different approach for this issue.
One possible solution is using application_name for pgmigrate connections search.
I've committed it as 1dcf5cb.
Could you check if pgmigrate from master branch is now correctly handles conflict termination in your setup?

No, I can't, because I don't work on that project anymore :(


Extremely large file and slow loading

Hey,
on startup i load a lot of xml files and store some information in maps.
I have tried using the 0.9.7 Version and was verry happy with it but when i switched to 1.0.6 i got problems regarding the performance.
I use this for creation:
DBMaker.newFileDB("D:/misc/db/map.db").closeOnJvmShutdown().make();
For the information of aproximately 3000 loaded xml files (the information i need not the files) the mapDB file needs 30mb of storage (on 0.9.7 )
I have stopped the startup when the mapDB file consumed more than 1GB of disk space and only about 10% of the total xml files had been loaded.
I have tried out all Version newer than 0.9.7, they all seem to behave the same in this matter.
Is there something wrong / changed so much that it is no longer usable for my use case?
Kind Regards

Closing, its on my todo list for MapDB 2.0


Consumers with lag hang on consumer .close

Problem
When we close a kafka cluster that has a consumer that has LAG, it hangs.
Following is a test that should passes but hangs instead:
(defexpect some-failing-test
  (let [consumer-clu (sut/kafka-cluster
                      {::sut/nodes "localhost:9092"
                       ::sut/shapes [(shape/topic (constantly "topic1")) (shape/edn) (shape/value-only)]
                       ::sut/consumer-config {:position :beginning}
                       ::sut/group-id "tester1"
                       ::sut/topics ["topic1"]})
        producer-clu (sut/kafka-cluster
                      {::sut/nodes "localhost:9092"
                       ::sut/shapes [(shape/topic (constantly "topic1")) (shape/edn) (shape/value-only)]})
        put-msg! (::sut/put! producer-clu)
        consume (::sut/consume consumer-clu)]
    (consume (fn [msg]
               (Thread/sleep 1000)
               (println msg)))
    (dotimes [_ 1000]
      (put-msg! "hello"))
    (.close producer-clu)
    (.close consumer-clu) ;; hangs here
    (expect "reached here")))

When the [::close] action is sent to the actor while the actor is polling, the putting-all function blocks:
(fn [events] ; function to handle events and resume
  (d/chain (ms/put-all! out-sink events) ;; out-sink has been closed
           #(when % (cmd-self [::resume duration]))))

Suggestion
It might be caused by the fact that (actor [::close])) in core/consumer is derefed:
(ms/on-closed out-sink (fn [] @(actor [::close])))

We can prevent the closing put to be be blocking by removing the deref.
(ms/on-closed out-sink (fn [] (actor [::close]))) ;; note that the `@` has been removed

It will pass the test but the messages that were being processed on close might be processed after the cluster has been close.
So in the test above, if consume is:
(consume (fn [msg]
               (println msg "1")
               (Thread/sleep 1000)
               (println msg "2")))

Then the results of the tests can be:
clj꞉robertluo.waterfall-test꞉>
; Running tests for the following namespaces:
;   robertluo.waterfall-test
;   robertluo.waterfall

hello 1
; 1 tests finished, all passing 👍, ns: 1, vars: 1
clj꞉robertluo.waterfall-test꞉>
hello 2
[::close]

We could (Thread/sleep poll-duration) after [::close] but it is not ideal:
(ms/on-closed out-sink (fn []
                             (actor [::close])
                             (Thread/sleep poll-duration)))

Using the future and still blocking the ::close put will make sure the consumer is closed on time
However, the last consumed messages might be processed after the consumer has closed preventing the offset commit.
In my test example from above, it means, I can have:
clj꞉robertluo.waterfall-test꞉>
; Running tests for the following namespaces:
;   robertluo.waterfall-test
;   robertluo.waterfall

hello 1
[::close]
; 1 tests finished, all passing 👍, ns: 1, vars: 1
clj꞉robertluo.waterfall-test꞉>
hello 2

Note that the second message was processed after the consumer was closed and therefore was not committed.

Can we use a promise, like this?
(defn- consumer-actor
  "..."
  [^Consumer consumer out-sink]
  (let [mailbox (ms/stream)
        ...
        closing-promise (promise) ; promise that gets delivered when consumer closes
        ...]
    (d/future
      (loop []
        (let [cmd @(ms/take! mailbox)]
          (if (= cmd [::close])
            (do (reset! closing? true) ; set closing flag
                (with-exception-handling "on closing consumer"
                  (.close consumer)) ; close consumer
                (deliver closing-promise true) ; attempts to dereference promise can now return
                ::stop) ; stop actor loop
            (do
              ...)))))
    {::actor cmd-self
     ::closing-promise closing-promise}))

(defn consumer
  [nodes group-id topics
   {:keys [poll-duration position]
    :as   conf
    :or   {poll-duration (Duration/ofSeconds 10)}}]
  (let [...
        {::keys [actor closing-promise]} (consumer-actor conr out-sink)]
    ...
    (ms/on-closed out-sink (fn []
                             (actor [::close]) ; avoid blocking putting close command into mailbox
                             @closing-promise)) ; wait for consumer to close
    ...))


Downstream clusters go into updating state when rancher server is upgraded from v2.5.9 to master-head

Rancher Server Setup

Rancher version: master-head 6c14c33
Installation option (Docker install/Helm Chart): Helm chart

If Helm Chart, Kubernetes Cluster and version (RKE1, RKE2, k3s, EKS, etc): RKE1

Proxy/Cert Details: self-signed

Information about the Cluster

Kubernetes version: v1.20.8  - Local
Cluster Type (Local/Downstream): Downstream infrastructure providers - k8s v1.19.13 and k8s v1.20.9

Describe the bug
When the rancher server is upgraded from v2.5.9 to master-head, the downstream clusters go into updating state.
To Reproduce

Create 2 downstream node driver clusters on k8s versions v1.19.13 and v1.20.9
Upgrade the rancher server to the latest master-head

Result
All downstream clusters go into updating state.
Expected Result
Downstream clusters should not be going into updating state.
Screenshots

Additional context
Related issue: https://github.com/rancher/rancher/issues/33749

Closing this ticket as it is currently not seen. Will re-open if it is seen again


Cell execution takes too long

Issue Type: Performance Issue
Of late, whenever I try and execute a cell on jupyter notebook on VS Code, the cell takes too long to execute.
A simple print hello statement, or a simple integer assignment to a variable takes 10-15 seconds. Note, this is the time it takes for the cell to start execution, not the total time taken to execute.
Once the cell starts to execute, it is very slow as well, but that's just my gut feeling.
Currently I am stuck on a simple list assignment (the list isn't large enough to overflow memory) and it's been 10 minutes but the cell still hasn't started execution.
VS Code version: Code 1.68.1 (30d9c6cd9483b2cc586687151bcbcd635f373630, 2022-06-14T12:48:58.283Z)
OS version: Windows_NT x64 10.0.19044
Restricted Mode: No
Remote OS version: Linux x64 5.4.0-1054-aws

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz (8 x 2808)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: disabled_offdirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_renderer: enabled_onvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
15.83GB (6.29GB free)

Process Argv
--crash-reporter-id 99e71ae7-6490-48f1-b1a0-10962d994c5f

Screen Reader
no

VM
0%

Item
Value

Remote
SSH: TM_Remote_Instance

OS
Linux x64 5.4.0-1054-aws

CPUs
AMD EPYC 7R32 (4 x 3248)

Memory (System)
15.46GB (0.90GB free)

VM
0%

Process Info
CPU %	Mem MB	   PID	Process
    0	    87	 32956	code main
    0	   203	  2752	   gpu-process
    0	    40	 16444	   utility-network-service
    0	   761	 20288	   window (● sent-analysis.ipynb - tm-ml-sentiment [SSH: TM_Remote_Instance] - Visual Studio Code)
    0	    44	 23268	   window (undefined)
    0	    71	 24188	   extensionHost
    0	    86	 24940	   issue-reporter
    0	    25	 30112	   crashpad-handler
    0	    74	 31052	   shared-process
    0	    58	 16548	     ptyHost
    0	     4	 23320	       C:\WINDOWS\System32\cmd.exe /c (type "C:\Users\Abdullah\AppData\Local\Temp\vscode-linux-multi-line-command---_-emote_-nstance-758612387.sh" | ssh -T -D 64915 "TM_Remote_Instance" bash) & exit /b 0
    0	    25	 32120	         ssh  -T -D 64915 "TM_Remote_Instance" bash
    0	     6	 31776	       console-window-host (Windows internal process)
    0	     6	 32508	       console-window-host (Windows internal process)
    0	     4	 36344	       C:\WINDOWS\System32\cmd.exe /c (type "C:\Users\Abdullah\AppData\Local\Temp\vscode-linux-multi-line-command---_-emote_-nstance-953024260.sh" | ssh -T -D 50447 "TM_Remote_Instance" bash) & exit /b 0
    0	     9	 27416	         ssh  -T -D 50447 "TM_Remote_Instance" bash
    0	    35	 23332	     fileWatcher

Remote: SSH: TM_Remote_Instance
CPU %	Mem MB	   PID	Process
    0	     0	  6554	remote agent
    0	     0	  6652	   ptyHost
    0	     0	 14773	   /bin/sh -c /bin/ps -ax -o pid=,ppid=,pcpu=,pmem=,command=
    0	     0	 14774	     /bin/ps -ax -o pid=,ppid=,pcpu=,pmem=,command=
    0	     0	 25920	   extensionHost
    0	     0	 25958	   fileWatcher

Workspace Info

|  Remote: SSH: TM_Remote_Instance|    Folder (tm-ml-sentiment): 9 files|      File types: ipynb(3) csv(2) md(1) bin(1)
|      Conf files:;

Extensions (6)

Extension
Author (truncated)
Version

jupyter-keymap
ms-
1.0.0

remote-ssh
ms-
0.82.1

remote-ssh-edit
ms-
0.80.0

jupyter
ms-
2022.5.1001601848

jupyter-keymap
ms-
1.0.0

jupyter-renderers
ms-
1.0.8

A/B Experiments
vsliv368cf:30146710
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythonvspyl392:30443607
vserr242:30382549
pythontb:30283811
vsjup518:30340749
pythonvspyt551cf:30345471
pythonptprofiler:30281270
vshan820:30294714
vstes263:30335439
vscoreces:30445986
pythondataviewer:30285071
vscod805:30301674
binariesv615:30325510
bridge0708:30335490
bridge0723:30353136
vsaa593:30376534
vsc1dst:30438360
pythonvs932:30410667
wslgetstarted:30449410
vscscmwlcmt:30465135
cppdebug:30492333
vsclangdc:30486549
c4g48928:30513948

Thanks for filing this issue.
Please could you:

Reload VS Code
Create a notebook & run a simple cell with the statment print("Hello World")
Then select the command Jupyter: Show Output and copy all of the output from the output panel & paste that information here (it congtain information such as vscode, jupyter extension, jupyter, python version information & the logs required to determine what's going on)

Here you go:
[12:01:27.667] Log Level: 2
[12:01:27.668]<EMAIL_ADDRESS>[12:01:27.668] win32 x64
[12:01:27.706] SSH Resolver called for "ssh-remote+7b22686f73744e616d65223a22544d5f52656d6f74655f496e7374616e6365227d", attempt 1
[12:01:27.706] "remote.SSH.useLocalServer": false
[12:01:27.706] "remote.SSH.showLoginTerminal": false
[12:01:27.706] "remote.SSH.remotePlatform": {"Jupyter":"linux","TM_Remote_Instance":"linux"}
[12:01:27.707] "remote.SSH.path": undefined
[12:01:27.707] "remote.SSH.configFile": undefined
[12:01:27.707] "remote.SSH.useFlock": true
[12:01:27.707] "remote.SSH.lockfilesInTmp": false
[12:01:27.707] "remote.SSH.localServerDownload": auto
[12:01:27.707] "remote.SSH.remoteServerListenOnSocket": false
[12:01:27.707] "remote.SSH.showLoginTerminal": false
[12:01:27.707] "remote.SSH.defaultExtensions": []
[12:01:27.708] "remote.SSH.loglevel": 2
[12:01:27.708] "remote.SSH.enableDynamicForwarding": true
[12:01:27.708] "remote.SSH.enableRemoteCommand": false
[12:01:27.708] "remote.SSH.serverPickPortsFromRange": {}
[12:01:27.708] "remote.SSH.serverInstallPath": {}
[12:01:27.711] SSH Resolver called for host: TM_Remote_Instance
[12:01:27.712] Setting up SSH remote "TM_Remote_Instance"
[12:01:27.751] Using commit id "30d9c6cd9483b2cc586687151bcbcd635f373630" and quality "stable" for server
[12:01:27.756] Install and start server if needed
[12:01:27.758] Checking ssh with "ssh -V"
[12:01:27.816] > OpenSSH_for_Windows_8.1p1
[12:01:27.817] > , LibreSSL 3.0.2

[12:01:27.826] Running script with connection command: ssh -T -D 51766 "TM_Remote_Instance" bash
[12:01:27.828] Terminal shell path: C:\WINDOWS\System32\cmd.exe
[12:01:30.903] > ]0;C:\WINDOWS\System32\cmd.exe
[12:01:30.904] Got some output, clearing connection timeout
[12:01:33.674] > a769b16ed217: running
[12:01:33.730] > Acquiring lock on /home/muhammad/.vscode-server/bin/30d9c6cd9483b2cc586687151bcb
> cd635f373630/vscode-remote-lock.muhammad.30d9c6cd9483b2cc586687151bcbcd635f37363
> 0
[12:01:33.741] > Found existing installation at /home/muhammad/.vscode-server/bin/30d9c6cd9483b2c
> c586687151bcbcd635f373630...
> Checking /home/muhammad/.vscode-server/.30d9c6cd9483b2cc586687151bcbcd635f373630
> .log and /home/muhammad/.vscode-server/.30d9c6cd9483b2cc586687151bcbcd635f373630
> .pid for a running server
> Looking for server with pid: 32290
[12:01:33.758] > Found running server...
>
> *
> * Reminder: You may only use this software with Visual Studio family products,
> * as described in the license (https://go.microsoft.com/fwlink/?linkid=2077057)
> *
>
[12:01:33.774] > Checking server status on port 34727 with wget
[12:01:33.792] > a769b16ed217: start
> SSH_AUTH_SOCK====
[12:01:33.937] > DISPLAY====
> webUiAccessToken====
> listeningOn==34727==
> osReleaseId==ubuntu==
> arch==x86_64==
> tmpDir==/run/user/1014==
> platform==linux==
> unpackResult====
> didLocalDownload==0==
> downloadTime====
> installTime====
> extInstallTime====
> serverStartTime====
> connectionToken==a1111111-11aa-1111-aa11-a1111a111111==
> a769b16ed217: end
[12:01:33.938] Received install output:
SSH_AUTH_SOCK====
DISPLAY====
webUiAccessToken====
listeningOn==34727==
osReleaseId==ubuntu==
arch==x86_64==
tmpDir==/run/user/1014==
platform==linux==
unpackResult====
didLocalDownload==0==
downloadTime====
installTime====
extInstallTime====
serverStartTime====
connectionToken==a1111111-11aa-1111-aa11-a1111a111111==

[12:01:33.939] Remote server is listening on 34727
[12:01:33.939] Parsed server configuration: {"serverConfiguration":{"remoteListeningOn":{"port":34727},"osReleaseId":"ubuntu","arch":"x86_64","webUiAccessToken":"","sshAuthSock":"","display":"","tmpDir":"/run/user/1014","platform":"linux","connectionToken":"a1111111-11aa-1111-aa11-a1111a111111"},"installUnpackCode":""}
[12:01:33.947] Starting forwarding server. localPort 51773 -> socksPort 51766 -> remotePort 34727
[12:01:33.948] Forwarding server listening on 51773
[12:01:33.948] Waiting for ssh tunnel to be ready
[12:01:33.950] [Forwarding server 51773] Got connection 0
[12:01:33.956] Tunneled 34727 to local port 51773
[12:01:33.957] Resolved "ssh-remote+7b22686f73744e616d65223a22544d5f52656d6f74655f496e7374616e6365227d" to "<IP_ADDRESS>:51773"
[12:01:33.981] ------

[12:01:34.019] [Forwarding server 51773] Got connection 1
[12:01:34.488] [Forwarding server 51773] Got connection 2
[12:01:46.037] [Forwarding server 51773] Got connection 3
[12:01:46.039] [Forwarding server 51773] Got connection
``` 4

Transferring to VS Code as this seems to be an issue with remote SSH connections (something we've had pref issues with in the past).
@rebornix /cc


[FLINK-13651][table-planner-blink] Blink planner should parse char(n)/varchar(n)/decimal(p, s) inside a string to corresponding datatype

What is the purpose of the change
Blink planner should parse decimal(p,s)/char(n)/varchar(n) inside a string to corresponding datatype
Brief change log

Refactor PlannerExpressionParserImpl for not using TypeInformation
Support parse decimal(p, s) inside a string to datatype
Support parse char(n) inside a string to datatype
Support parse varchar(n) inside a string to datatype

Verifying this change
This change is already covered by existing tests, such as (ScalarFunctionsTest).
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (yes / no)
The public API, i.e., is any changed class annotated with @Public(Evolving): (yes / no)
The serializers: (yes / no / don't know)
The runtime per-record code paths (performance sensitive): (yes / no / don't know)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes / no / don't know)
The S3 file system connector: (yes / no / don't know)

Documentation

Does this pull request introduce a new feature? (yes / no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@JingsongLi @wuchong Could you take a look at this?


Render "jumped" when scroll up and down in viewer.html at android phone

After testing in android+SafariMobile,  when scroll up or down, viewer.html rendered jumping, or to say, the frame rendering behind my touch up event. So the display jumped.
It's not good experience in mobile.
Plus, need punch in and out gesture to get the PDF zoom in and out, and minimum the UI will be better.

Pinch to zoom is tracked in #2582. I do not experience that jumping effect on my Nexus 6 (Android) too much. It's probably related to performance issues during rendering. We have been talking about creating a separate Android/mobile viewer that would make the experience on mobile devices a lot better, but work has not started on that yet as far as I know. Since there are no concrete points to fix here, other than improving the general performance of PDF.js and creating a separate Android viewer, and the pinch to zoom issue is already tracked elsewhere I'm closing this issue.


Please Add these Porn Sites

Hello @StevenBlack and @Sinfonietta,
Here are more porn sites to add to the filter:
epawg.com
en.vidmo.pro
fullporner.com
2chen.moe
8chan.moe
nutchaser.com
erothots.co
4hdxxx.com
nudeleakers.com
cambabes.tube
celebforum.co
celeb.to
jerkmate.tv
fapnation.com
flashthepublic.com
hdsupersex.com
xxteenhub.com
bestvideoleak.com
newdayporn.com
newporntv.me
goodnightporn.com
xvideoleaks.com
goldengirlporn.com
whereismyporn.com
nicetitties.net
javtv.cc
javtv.to
javtc.net
njav.tv
njavtv.com
beeg.day
beeg.yachts
jav.casa
anybunny.casa
sexfreemovie.fun
javshy.one
javcrave.com
javpornfull.com
javvideoporn.com
javvideos.porn
xxxtube.porn
I will append this list if I find more sites to add to the blocklist. I'm thankful for your work!

Thank you @g1t3m.  Ping @sinfonietta thanks.

@StevenBlack I have added all of these.

Thank you both for adding these sites. Here are a few more I came across:
wetqueen.com
dankwank.net
pornito.xxx
pornfaze.com
danielleftv.com
leaked.fans
thothub.org

Thank you @Sinfonietta closing.


logging: Add --with-log-level=LEVEL parameter to configure build.

Allows simple build time configuration of logging level by passing --with-log-level=LEVEL.
LEVEL may equal any legal numerical values (0,1,2,3,4) or be passed via human readable strings (none,crit,critical,warn,warning,info,debg,debug).
When not passed, the default level from config header is used.

Codecov Report

Merging #1698 into master will decrease coverage by 0.19%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master    #1698     +/-   ##
=========================================
- Coverage   67.35%   67.16%   -0.2%
=========================================
  Files         171      171
  Lines       21281    21279      -2
  Branches     2610     2610
=========================================
- Hits        14334    14291     -43
- Misses       6033     6080     +47
+ Partials      914      908      -6

Impacted Files
Coverage Δ

src/core/coap/coap_client.hpp
73.33% <0%> (-20%)
:arrow_down:

src/core/coap/coap_client.cpp
52.38% <0%> (-19.81%)
:arrow_down:

src/core/coap/coap_server.cpp
81.88% <0%> (-7.25%)
:arrow_down:

src/core/coap/coap_server.hpp
89.65% <0%> (-3.68%)
:arrow_down:

src/core/net/ip6.cpp
63.44% <0%> (-2.36%)
:arrow_down:

src/core/net/ip6_mpl.cpp
93.07% <0%> (-1.54%)
:arrow_down:

src/core/net/udp6.cpp
82.85% <0%> (-0.96%)
:arrow_down:

src/core/meshcop/joiner_router.cpp
83.5% <0%> (-0.5%)
:arrow_down:

src/core/thread/network_data_leader.cpp
84.46% <0%> (-0.49%)
:arrow_down:

src/core/common/message.cpp
95.86% <0%> (-0.46%)
:arrow_down:

... and 11 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 41b82d3...e5cab55. Read the comment docs.

@turon, what is the need/reason for this change?
We currently have all  the log related settings here: https://github.com/openthread/openthread/blob/master/src/core/openthread-core-default-config.h#L416

@abtink Yes, this PR allows you to override the default in the configuration header at build time as a parameter to configure.  This allows you to easily build development vs. release builds for example from the top-level configure command (similar to the selection of ftd vs mtd).

@turon why do we need this since the same setting already exists in openthread-core-config.h?
I prefer that we have a single way for specifying a parameter. I am concerned that having different ways which can override each other can be confusing and cause usage errors.

@abtink I've found it to be a fairly common use case for me as a developer to want to rebuild with a higher log level without touching any of the actual source files.  The log level remains in the purview of openthread-core-config.h when --with-log-level is not passed.  Also, the developer may forcibly hard-code their desired log level in a platform-specific openthread-core-config.h without ifndef if they so choose.
Similar to the introduction of otGetDynamicLogLevel, I think this PR makes the logging facility more configurable and convenient to use with minimal downside risks.  I would argue there is far more to get confused by in the logging design than whether this config option or the header is making the final log-level decision in the build.
Any others that support this PR?

Closing stale PR.


Decoupling: Make dashboard API Callbacks optional

Context
While working on https://github.com/google/web-stories-wp/pull/9644, we realized that we have to provide some default API callbacks for the dashboard to work here https://github.com/google/web-stories-wp/blob/ad567f6777ac9d419029b184b7f671f100909609/.storybook/stories/playground/dashboard/index.js#L38-L45
To simplify the integration layer, we should make these optional.
Summary
In this PR, we have tried to do two things

Do not render the related button or link if the underlying API callback is not provided.
If possible, handle the case when the function is directly called elsewhere like inside some useEffect

[x] Make getAuthor optional ( https://github.com/google/web-stories-wp/pull/9752/commits/1f3140bf5b5a079be0d13c414caaf349d73eb273 )
[x] Make createStoryFromTemplate optional ( https://github.com/google/web-stories-wp/pull/9752/commits/9b503829866345ae604b6e6a324cb0bdbff9140e )
[x] Make duplicateStory, trashStory, updateStory optional ( https://github.com/google/web-stories-wp/pull/9752/commits/fa5626b7f70f42e6c544f6e7f6a130a78c7f0c29 )
[x] Update unit and integration tests for this change.

Relevant Technical Choices
If checking conditions using the API callbacks doesn't make a lot of sense, just a rough idea that perhaps we may introduce some more custom capabilities and use those as conditionals to not render those buttons and links.
To-do

User-facing changes

Testing Instructions

[x] This is a non-user-facing change and requires no QA

This PR can be tested by following these steps:

Reviews
Does this PR have a security-related impact?

Does this PR change what data or activity we track or use?

Does this PR have a legal-related impact?

Checklist

[ ] This PR addresses an existing issue and I have linked this PR to it in ZenHub
[x] I have tested this code to the best of my abilities
[x] I have verified accessibility to the best of my abilities (docs)
[x] I have verified i18n and l10n (translation, right-to-left layout) to the best of my abilities
[x] This code is covered by automated tests (unit, integration, and/or e2e) to verify it works as intended (docs)
[x] I have added documentation where necessary
[x] I have added a matching Type: XYZ label to the PR

Partially addresses #456

Karma "MediaPane fetching should fetch 2nd page" is also failing on other PRs like https://github.com/google/web-stories-wp/pull/9764 , probably after this PR https://github.com/google/web-stories-wp/pull/9757/files

All passed 🎉

@swissspidy If no other feedback, it's ready to be merged.


Error mounting url

Hello everyone!
I have been playing around with autopkg the passed couple days and would love to get it working.
It looks like everything is working until it has to talk to the JSS. It mounts our CasperShare but give me this error.
Error mounting Url "smb://jssuser:jsspassord:@ourjss:139/caspershare$: None
defaults read com.github.autopkg JSS_REPOS - looks right
defaults read com.github.autopkg JSS_URL - looks right
defaults read com.github.autopkg JSS_VERIFY_SSL - says no SSL exists
defaults read com.github.autopkg API_USERNAME - looks good
defaults read com.github.autopkg API_PASSWORD - looks good

Hi Chris,
So by "looks right", what do you see when you defaults read /Library/Preferences/com.github.autopkg JSS_REPOS? Obviously remove any URL's or anything that references private information, but I would like to see the actual output.

Output
(
{
name = Name of casper distribution point;
Password = Password for read/right account for that distribution point
   }

)

What is that $ doing at the end... Is there a newline also? Or is that just AutoPkgr wrapping the text?
Given the URL output in the AutoPkgr error message, can you mount that share in Finder (Connect to Server) or on the commandline?
Is it definitely port 139 and not 445?
Does the password have any XML or bash reserved characters in it that may be getting encoded incorrectly?

Thank you for helping out shegcraig!
We have the $ in our smb share name just to hide it
I can manually mount the Distribution point no problem. But also when i run the AutoPkg recipes it mounts the Distribution point.
And it for sure is on port 139
our password does not have any XML or bash reserved characters.

So AutoPkg will mount the distribution point, but not AutoPkgr?

no they both mount the distribution point. That is the last step it completes before giving me that error.

We figured it out! does not respect the $

That's super weird that it actually mounts, but it still has issues. The mount used to be done by shelling out to the mount_smbfs command, but it does it at a lower level now, so I'm surprised it has issues. But everything is good now?

Yes its seems to be working without the $ in the distribution points name. It is able to read/write to our JSS. And might i say AutoPkg is AWESOME! thank you!

Hi Guys, I have the same problem, however I do not have a $ in my share. AutoPkg mounts the drive and I can read and write to it. But I get the same error message. I am mounting a afp, and I am running this on a 10.11.3 machine.

Hi @bytesman
So you get the "error mounting URL" error, but it actually mounts it?

Yes the drive is mounted and I can write into it. I disconnect it and then AutoPkgr/JSSImporter remounts it if I run the update process.

I believe @Chriskmpruitt is all set here. @bytesman are you still trying to get this working, or are you all set?

Yes even with the new version still the same problem.
On Wed, 23 Mar 2016 at 15:01, Shea Craig<EMAIL_ADDRESS>wrote:

I believe @Chriskmpruitt https://github.com/Chriskmpruitt is all set
here. @bytesman https://github.com/bytesman are you still trying to get
this working, or are you all set?
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/sheagcraig/JSSImporter/issues/74#issuecomment-200356840

@Chriskmpruitt How were you able to resolve this?  Did you just change your distribution point to not have a "$" at the end, or is there another workaround?

@cjduckk  Yes when trying to mount the distribution point, it was not respecting the $ in the name. Change the Share name and now autopkg connects without a problem. 👍

I am also having this error in AutoPkgr version 1.4.1(1308). Manual mount to smb share works fine, but when running recipes reports
"Error mounting url "smb://casperusername:pass@IPOFSERVER:PORT/Distribution_Share": None"
I have checked that the share is not mounted, and that the information in the JSS_REPOS variable is correct.
The recipe creates the pkg file successfully - it is only at the connection to JSS where it fails.
The machine I am running it on is OS X 10.11.6 and is not managed by Casper. The Distribution Share is not hidden by $

I am seeing this same problem to our dev server (seeing different issue in prod). Has anyone figured out a solution? Also not using $ in the share name.

OP issue was a $ in the share name.
Other folks who have commented on here, an update:
JSSImporter testing branch will only mount shares that aren't already mounted, and it won't unmount shares that were mounted prior the JSSImporter run. SO, at least to get working, mount your shares and carry on.
If JSSImporter still can't mount your shares, get on the Macadmins Slack and ask around there, and include your config file and/or open an issue with your exact issue here and let's all help each other out.
Frequent issues tend to be using a different port than normal SMB; the automatic DP setting detection is just pulling data from however it is configured on the JSS. Even if Casper Admin can mount it fine, doesn't mean it's going to translate.
Passwords with certain characters may be an issue, although we're not shelling out to mount so I hope that's not an issue.


Unable to set up server using snapshot 20w45a

Hey guys I am trying to spin up a minecraft server with the newest snapshot 20w45a but it seems to only start in 1.16.4. Below is my docker-compose.yaml file, and I was hoping you guys could take a look and see where I messed up.
version: "3"

services:
  # The main survival world
  survival:
    container_name: ${CONTAINER_NAME}_survival
    image: itzg/minecraft-server
    ports:
      - ${SURVIVAL_PORT}:25565
    volumes:
      - ${VOLUME}/survivalData:/data
    environment:
      EULA: "TRUE"
      SERVER_NAME: ${SERVER_NAME}
      MOTD: ${MOTD}
      VERSON: ${IMAGE}
      TZ: ${TIME_ZONE}
      DIFFICULTY: ${DIFFICULTY}
      ANNOUNCE_PLAYER_ACHIEVEMENTS: ${ANNOUNCE_PLAYER_ACHIEVEMENTS}
      ENABLE_COMMAND_BLOCK: ${ENABLE_COMMAND_BLOCK}
      FORCE_GAMEMODE: ${FORCE_GAMEMODE}
      MAX_BUILD_HEIGHT: ${MAX_BUILD_HEIGHT}
      SEED: ${SEED}
      ENABLE_AUTOPAUSE: ${ENABLE_AUTOPAUSE}
      MAX_MEMORY: ${MAX_MEMORY}
      # MODE: creative
      ENABLE_RCON: "true"
      RCON_PASSWORD: "testing"
      RCON_PORT: 28016
      # enable env variable replacement
      REPLACE_ENV_VARIABLES: "TRUE"
      # define an optional prefix for your env variables you want to replace
      ENV_VARIABLE_PREFIX: "CFG_"
      # and here are the actual variables
      CFG_DB_HOST: "http://localhost:3306"
      CFG_DB_NAME: "minecraft"
      CFG_DB_PASSWORD_FILE: "/run/secrets/db_password"
    restart: always

And I am starting that with a bash script that looks like this;
SERVER_NAME="Universe"
MOTD="to first bake a pie you must invent the universe..."
IMAGE="SNAPSHOT"
VOLUME="./serverData"
TIME_ZONE="America/Fort_Wayne"
SURVIVAL_PORT="33333"
CREATIVE_PORT="33335"
ENABLE_AUTOPAUSE="TRUE"
MAX_MEMORY="3G"
DIFFICULTY="normal"
ANNOUNCE_PLAYER_ACHIEVEMENTS="false"
ENABLE_COMMAND_BLOCK="false"
FORCE_GAMEMODE="false"
MAX_BUILD_HEIGHT="256"
SEED="7185855458401771874"

CONTAINER_NAME=${SERVER_NAME} \
MOTD=${MOTD} \
IMAGE=${IMAGE} \
VOLUME=${VOLUME} \
TIME_ZONE=${TIME_ZONE} \
SURVIVAL_PORT=${SURVIVAL_PORT} \
ENABLE_AUTOPAUSE=${ENABLE_AUTOPAUSE} \
MAX_MEMORY=${MAX_MEMORY} \
DIFFICULTY=${DIFFICULTY} \
ANNOUNCE_PLAYER_ACHIEVEMENTS=${ANNOUNCE_PLAYER_ACHIEVEMENTS} \
ENABLE_COMMAND_BLOCK=${ENABLE_COMMAND_BLOCK} \
FORCE_GAMEMODE=${FORCE_GAMEMODE} \
MAX_BUILD_HEIGHT=${MAX_BUILD_HEIGHT} \
EULA=${EULA} \
SEED=${SEED} \
docker-compose up

I am using the batch script to set the environment variables instead of a .env file because you cannot set things like container name or ports with that, unless you guys know a work around too haha!
But thanks for looking guys I really appreciate it!

It took a couple times looking at your snippet, but you have a typo on the environment line:
      VERSON: ${IMAGE}

where it should be VERSION.

 Thank you so much haha!


Input doesn't show up in terminal after running diff command using bash

VSCode Version: Code 1.12.2 (19222cdc84ce72202478ba1cec5cb557b71163de, 2017-05-10T13:20:36.315Z)
OS Version: Windows_NT ia32 10.0.14393
Extensions:

Extension
Author
Version

jshint
dbaeumer
0.10.15

Steps to Reproduce:

Open VS Code
Open Terminal to use bash.exe
git diff branch1 branch2
ctrl + c to exit diff
Type characters into terminal (they won't appear)

I think I fixed this today, could you try reproduce in the Insiders build tomorrow?

Sure, I'll check it out. Not familiar with the process of getting the most recent version, but should be able to figure it out. Thanks!


Remove the move funds modal from hide wallet

Remove the move funds modal when hiding a wallet, and allow deleting watch only wallets
Addresses: #5330 #5590

Yes when deleting we should present the warning (I think for now we do only if it has funds, which is fine for now)


Plot data getting stored in database inadvertently

"Sometimes" Bokeh plot data is getting stored in the database directly; this hasn't been a problem until now as plots have mostly been very small, but we are now hitting the MongoDB limits in #473.

Just to extend this discussion a bit; the UI still sends the entire plot state back to the API when a block is updated. We could either
a) disable this and potentially save a chunk of network calls
b) re-enable storing plots in the database when the plot is sufficiently small [I think this is problematic for the future case that the plot then stops other data from being stored...]


[Bug]: User createdAt trait not getting added in flagsmith user identifier

Is there an existing issue for this?

[X] I have searched the existing issues

Description
The flagsmith user createdAt trait is not getting added to the user identifiers.
Steps To Reproduce

Signup with a new user on app.appsmith.com
Check the flagsmith dashboard for the newly created user traits.
All the other traits are added but createdAt trait is missing

Public Sample App
No response
Environment
Production
Issue video log
No response
Version
v1.9.29-SNAPSHOT

Paired up with @NilanshBansal to Verify it on release and looks good to me


Proposal: containerize the development environment

This is to lower the barrier to entry for hacking on Deis while also standardizing the development environment for all contributors.

@krancour Right now I'm building everything on single make target for each builder type.
In future once things gets settled down with builder I will make proper targets for docker build and docker push. let me know if you have any idea

There are a lot of ideas in deis/deis#4351, which this original proposal came from (just formalizing it in each repository).

@krancour I believe this is done. The Makefile's build and test targets execute everything inside of quay.io/deis/go-dev:0.3.0 containers.
Closing. Can you reopen this if you think there's more to be done?

@arschles :100:


Bug with TilingSprite anchor behaviour

Changing the anchor of a TilingSprite causes the tiling to be misplaced in 4.x using WebGL.
In 3.x or 4.x with Canvas, the Tiling would stay the same. The sprite would be positioned and rotated relative to the anchor like one would expect.
Drawing a circle TilingSprite in 4.x with WebGL with anchor at (0.5,0.5):

Drawing a circle TilingSprite in 3.x (or 4.x with Canvas) with anchor at (0.5,0.5):

This behaviour is important when drawing textures such as a rope.
Behaviour in 4.x WebGL:

Comment/uncomment

Thanks for this report, definitely looks like a bug.  I reduced the fiddle a bit, pointed it at latest and included both renderers; https://jsfiddle.net/mb6b808c/

Interesting that at anchor.set(0) and anchor.set(1) canvas and webgl both match up. It's the bits in between where it goes wrong


Expose RX EOF bit in circular DMA transfers

The current API for a circular rx transfer is here:
https://github.com/esp-rs/esp-hal/blob/eddcb7354f75d9eecc8f356140fdd0feac9809b6/esp-hal/src/dma/mod.rs#L2099-L2150
It allows you to read bytes from the DMA buffer and check how many are available to read.
Some peripherals like LCD_CAM, PARL_IO and I2S emit an EOF bit along with the data that signifies the end of a "packet".
This EOF bit (often called suc_eof) is available in the DMA descriptor and when the DMA hardware sets it after receiving an EOF bit/marker from the peripheral, it closes the active descriptor and jumps to the next one.
This issue is asking for DmaTransferRxCircular to allow the user to check if and where this EOF happens.
This is important in LCD_CAM for users of the camera driver to know where one frame/picture ends and another starts.
(This may be useful in other drivers too but I haven't checked)

My personal need for this has been solved by #2242.


Explicit init vs. lazy init of thread server

I am sure there is some legacy knowledge here that I don't have. So hoping one of the maintainers can elaborate.
Question: Why is blas_thread_init() called via the constructor mechanism in driver/others/memory.c? While at the same time there is code inside exec_blas_async() to evaluate whether the server has been started and call blas_thread_init() if needed. If we always init at startup, is there a use-case scenario where the server is shutdown and then later restarted?
The concern I have is that blas_thread_init() allocates a default number of threads (ie. number of cpu cores) before any possible call to openblas_set_num_threads(). If I have a 64 core machine, then 64 threads are created even my code intends to limit usage to only one thread/core. This requires the worker_thread code path to deal with an odd case where the actual number of worker threads exceeds the number of threads we ever intend to use.
What I'd like to do is avoid paying the cost of an initialization check on every submission of work if we can safely guarantee that the server is always initialized on startup. But obviously that is not possible if there are valid startup-shutdown-startup patterns.
Alternatively I'd prefer to kill any extra worker threads when an openblas_set_num_threads() call reduces the number of threads. More threads can be created as needed.
Any thoughts? I have an amended PR ready to go for the latter strategy in blas_server_win32.c

Initializing in the constructor before any openblas_set_num_threads can be seen or processed is IMO unavoidable for shared libraries. The startup-shutdown-startup scenario is relevant (only) when the calling program does a fork(). Killing unneeded workers has been experimented with in the past, but IIRC the overhead turned out to be prohibitive at least in certain OpenMP settings. None of this may be particularly relevant in your Windows scenario.

Couldn't this be solved, even for shared libraries, by having an explicit openblas_init() call which could in turn call blas_thread_init()? This would allow for initialization and resource allocation to be deterministic rather than defined by shared library load time. The call to openblas_set_num_threads() already calls blas_thread_init(). Allowing that pattern would keep the library from creating unused threads.
But maybe I am missing something subtle as I've only just started thinking about this one.


Misleading docs for ssl-passthrough

The ssl-passthrough docs say:

The annotation ingress.kubernetes.io/ssl-passthrough allows to configure TLS termination in the pod and not in NGINX. This is possible thanks to the ngx_stream_ssl_preread_module that enables the extraction of the server name information requested through SNI from the ClientHello message at the preread phase.

The bit about preread is not true since de14e2f4. Maybe the second sentence should just be removed.

@drigz thank you for this. I will open a PR to remove this from the docs.