Poor DDEV performance (macOS MBP 2013)

Everything is really slow on my Mac. TYPO3 Frontend and Backend take at least 10 seconds to load. You cannot really work with it. The same project on a Windows Machine is super fast.
MacBook Pro (Retina 13 Zoll, Anfang 2013)
2,6 GHz Intel Core i5
8 GB 1600 MHz DDR3
SSD Drive 256 GB  / 50 GB free
Mac OS Mojave.
Question: How can I debug this?
Here is my config file:
APIVersion: v1.4.1
name: myproject
type: typo3
docroot: public
php_version: "7.2"
webserver_type: nginx-fpm
router_http_port: "80"
router_https_port: "443"
xdebug_enabled: false
additional_hostnames: []
additional_fqdns: []
provider: default
# This config.yaml was created with ddev version v1.4.1
# webimage: drud/ddev-webserver:v1.4.0
# dbimage: drud/ddev-dbserver:v1.4.0
# dbaimage: drud/phpmyadmin:v1.4.0
# However we do not recommend explicitly wiring these images into the
# config.yaml as they may break future versions of ddev.
# You can update this config.yaml using 'ddev config'.

# Key features of ddev's config.yaml:

# name: <projectname> # Name of the project, automatically provides
#   http://projectname.ddev.local and https://projectname.ddev.local

# type: <projecttype>  # drupal6/7/8, backdrop, typo3, wordpress, php

# docroot: <relative_path> # Relative path to the directory containing index.php.

# php_version: "7.1"  # PHP version to use, "5.6", "7.0", "7.1", "7.2"

# You can explicitly specify the webimage, dbimage, dbaimage lines but this
# is not recommended, as the images are often closely tied to ddev's' behavior,
# so this can break upgrades.

# webimage: <docker_image>  # nginx/php docker image.
# dbimage: <docker_image>  # mariadb docker image.
# dbaimage: <docker_image>

# router_http_port: <port>  # Port to be used for http (defaults to port 80)
# router_https_port: <port> # Port for https (defaults to 443)

# xdebug_enabled: false  # Set to true to enable xdebug and "ddev start" or "ddev restart"

# webserver_type: nginx-fpm  # Can be set to apache-fpm or apache-cgi as well

additional_hostnames:
  - myproject2
# would provide http and https URLs for "somename.ddev.local"
# and "someothername.ddev.local".

# additional_fqdns:
#  - example.com
#  - sub1.example.com
# would provide http and https URLs for "example.com" and "sub1.example.com"
# Please take care with this because it can cause great confusion.

# upload_dir: custom/upload/dir
# would set the destination path for ddev import-files to custom/upload/dir.

# working_dir:
#   web: /var/www/html
#   db: /home
# would set the default working directory for the web and db services.
# These values specify the destination directory for ddev ssh and the
# directory in which commands passed into ddev exec are run.

# omit_containers: ["dba", "ddev-ssh-agent"]
# would omit the dba (phpMyAdmin) and ddev-ssh-agent containers. Currently
# only those two containers can be omitted here.

# provider: default # Currently either "default" or "pantheon"
#
# Many ddev commands can be extended to run tasks after the ddev command is
# executed.
# See https://ddev.readthedocs.io/en/stable/users/extending-commands/ for more
# information on the commands that can be extended and the tasks you can define
# for them. Example:

You're saying it's faster with ddev on Windows? I've never ever heard that before. Windows is normally significantly slower.
Please reset your docker to factory defaults and see what happens. You can do that safely if you don't have any non-ddev containers or volumes you're counting on.

OK, now first thing that happens:
ERROR: for ddev-myproject-db  Cannot create container for service db: invalid mount config for type "bind": bind source path does not exist: /Applications/MAMP/myproject/.ddev/import-db
So what I did last time to get rid of the message, is add the folder to the shared folders in docker. Is this correct?

Docker and nearly every other tool assumes that you put a project directory in your home directory.
But if you really want to run things out in the /Applications area (highly discouraged, that's not the way it should work for many reasons) then you'll have to share /Applications in file sharing in your docker config.

OK, set up a new project in ~/projects/myProject. No binding errors, but still super slow. I am running DDEV 1.5.0. How can I debug this? How can I find out where the bottleneck is between docker and DDEV?

Could you please set up a simple basic TYPO3 project using composer (if that's not what you've already done)? https://ddev.readthedocs.io/en/latest/users/cli-usage/#typo3-quickstart
See what that does.

I'd recommend temporarily turning off any virus checker or firewall that you might have going, and just as a regular paranoid step, reboot your machine.
Like I say I have never ever seen a Windows machine faster than Mac.

This is just a regular filesystem on a regular mac right?  You say "SSD Flash Drive 256 GB" but I assume you just mean it's on a regular SSD, not on a flash drive. If it's on a flash drive that's never going to work out.

Could you please set up a simple basic TYPO3 project using composer (if that's not what you've already done)? https://ddev.readthedocs.io/en/latest/users/cli-usage/#typo3-quickstart
I did just that: https://streamable.com/05jzs

This is just a regular filesystem on a regular mac right?
Yes, regular filesystem, nothing fancy. It's a SSD, correct.

Looks painful! A 2013 mac is getting old of course, but I used a 2013 MacBook Air until recently.   I assume your Windows machine was newer.
You are welcome to try out the webcache feature we just introduced, https://www.drud.com/ddev-local/ddev-locals-new-webcache-feature/ - you might be really happy with it.
But I do wish we knew what was going on for you.
Oh, you don't say... but you're using Docker-for-Mac right? Not Docker Toolbox? Could you give a ddev version please?

I am having the same issue - every page load is slow as molasses.  The Drupal 8 install was painfully slow - 20+ minutes.  Once installed, page loads are taking 6-8 seconds each.  Drupal 8 running on MAMP Pro with no caching is snappy.
I'm also on a 2013 Mac but it is still blazing fast with most other operations.
2.8 GHz Intel Core i7
16 GB DDR3 memory
750 GB SSD with about half free
Docker was installed via the Docker for Mac dmg.
My config file (everything else in the file is commented out):
APIVersion: v1.5.0
name: probirt.d8.local
type: drupal8
docroot: web
php_version: "7.1"
webserver_type: nginx-fpm
router_http_port: "80"
router_https_port: "443"
xdebug_enabled: false
additional_hostnames: []
additional_fqdns: []
mariadb_version: "10.2"
webcache_enabled: false
provider: default
Otherwise, getting up and running with ddev was a dream.  I'm hoping I can get this resolved so that I can actually be productive with it.

Oh, you don't say... but you're using Docker-for-Mac right? Not Docker Toolbox? Could you give a ddev version please?

It's just the normal Mac installer, Docker Desktop Community.
It is set to Factory Reset, docker settings not changed.

$ ~/projects/my-typo3-site> ddev version
bgsync        		drud/ddev-bgsync:v1.5.0
cli	          	v1.5.0
commit        		v1.5.0
db            		drud/ddev-dbserver:v1.5.0-10.2
dba           		drud/phpmyadmin:v1.5.0
ddev-ssh-agent		drud/ddev-ssh-agent:v1.5.0
docker        		18.09.0
docker-compose		1.23.2
domain        		ddev.local
router        		drud/ddev-router:v1.5.0
web           		drud/ddev-webserver:v1.5.0

I used the guide here to set up a basic TYPO3 example page:
https://ddev.readthedocs.io/en/latest/users/cli-usage/#typo3-quickstart
I used the installer script to install DDEV:
https://ddev.readthedocs.io/en/latest/#installationupgrade-script-linux-and-macos

Docker Desktop and ddev version exactly the same as OP:

Also used the same install but for Drupal 8:
https://ddev.readthedocs.io/en/latest/users/cli-usage/#drupal-8-quickstart
I used the homebrew install:
https://ddev.readthedocs.io/en/latest/#homebrew-macos

That's about right for installing Drupal on MAMP for me - the 7ish minutes.  The Drupal install on ddev was 3 times that though.  Either way, I can live with 20 minutes installs as it's not something I need to do everyday.  But 10 second page loads makes it pretty unusual for daily development.  Hoping you can reproduce with your 2013 MBP.

Hope you'll try the webcache_enabled while you're at it as well, would love to know how it works for you. @micnap you did try resetting docker to factory defaults?

Just adding that I too am finding performance quite slow on a 2013 i7 MBP.
I also have DDEV running on a Ubuntu 18.04 machine with almost identical CPU and likewise 16GB memory and I am finding the difference between the two setups to be very noticeable.
I understand there are file access issues with Docker on Mac and Windows versus Linux, but this issue make me wonder if something else might be in play.

Linux is crazy faster, always. Linux is massively faster even when running in a virtual machine on MacOS :) So that's not a good comparison. But I do wonder what might be going on, and if it's only on older macs. I'll get out the 2013 MBP we have and relieve it from its testbot duties hopefully later today.

Oh, Could each of you please update your reports to include the filesystem you're using? People who have reinstalled recently have APFS. If you have "MacOS Extended", that could be a significant difference. And I remember that Docker made a major change in behavior when APFS came along.

APFS

Oh, interesting, I note that this MBP2013 (tests still in progress) is still High Sierra, so that gives us a little opportunity to see if that's what has happened here.

I should say that I just bought this (used) MBP only a few days ago. So it has a brand new install of High Sierra.

Here's what I see on our MBP 2013, it's pretty painful, I'll admit. Note that after poking around, I turned on webcache_enabled: true to show you the result. It's mighty nice. It is experimental as explained in the blog post

Woot!  That did it for me.  webcache_enabled set to true makes it load quickly like I'm used to.
For reference, I'm also on Sierra (10.12.5) and my file system is Logical Volume.
THANK YOU!

It's super nice, but be watching carefully. 2-way sync like this is experimental, but the feeling is fantastic, isn't it. Please, if you have any problems with it, let us know what happened and please try hard to figure out if you can recreate it. For example, if I bring up a project and then suddenly rm -r the entire vendor directory, it will in fact break the syncing, and I might even have to composer install as a result.

Wow. I installed the Droopler distribution, which has quite a number of modules enabled. Loading the module page without webcache took an average 15 seconds. With webcache, about 1.6 seconds.
Regarding your comment above Randy. Would 'ddev remove' protect against your example of how syncing might get broken?

So ddev rm and ddev start is pretty much a cure for putting anything on the host that ought to be in the container, easy recovery if something does go wrong. The problem is if something important happens in the container and doesn't get properly propagated back to the host. In that case, the contents of the host will not be fully consistent.  So for example, if you do a massive ddev composer remove, which removes all kinds of things from vendor inside the container, and then syncing fails during that process, the result won't make it onto the host, and then the remove/start combination wont help.  Another example, although less likely, would be a webserver activity (uploading files?) that changed a lot of stuff and then syncing failed. It would never make it back onto the host.
Mostly when you do something on the host, you're doing it on purpose, and it doesn't matter if it somehow doesn't reach consistency inside the container; in that case remove/push is a total solution.
ddev logs -s bgsync -f is fascinating (for a little while) and will tell you what's actually going on.

webcache_enabled: true helped me too. still weird though, that the only the mbp2013 has issues here.

Is this so? No issues with later Macs

I did some study on this and found that in ddev v1.5.0 we had a performance regression related to how we mount the code into the docker container. I think you'll find that v1.4.1 performs better for you, and we'll try to get a v1.5.1 solving this problem in your hands pretty soon.

@rfay I'm upgrading to 1.5.0 now and testing out the web caching feature in hopes of finally getting over the Mac OS Docker file mounting issues that have kept me from using Docker-based dev solutions on large projects, but is there a way I can add this line somewhere until 1.5.1 is released?
Since it's just a small change, I'd like to see how it adds to the performance difference between 1.4.1 and 1.5.0.
I installed/updated via Homebrew and I see the ddev.rb file but I've never tried messing with Homebrew outside of that process.

So with web caching enabled in my test of a clean admin/config after clearing cache on a D7 site went from 12 - 15 second response to 4 - 5 second response. Very cool and satisfying for now!

Yes, I find it very satisfying as well. There's more detail at https://www.drud.com/ddev-local/ddev-locals-new-webcache-feature/ if you haven't read/watched that; there are caveats there for you; it's an experimental feature and we want to hear your results and any problems you have.

@rfay also slow on my 2018 mbp will try webcache. Have you considered selective caching? Means that only some folders (like vendor) are cached and others are still bin mounted?
If this is configurable it would make live a lot easier 😉 and ddev faster as well.

Selective caching? This sounds interesting.
One problem I am finding with webcache_enabled is when theming with a browser refresh tool like BrowserSync. Such applications watch scss/css/js files for changes and automatically reload the browser for previewing after a change is detected. As well as BrowserSync there are similar others, including LIve Reload and Prepros
BrowserSync not seem to notice file changes on the host immediately and it seems necessary save a file twice, which breaks the workflow.
In https://github.com/drud/ddev/issues/1363#issuecomment-451101716 @kaystrobach seems to be seeing something similar.

Yes, changing files with webcache enabled does not seem to trigger the sync always. But selective caching would speed up work a lot.
And as written in #1363 i would prefer using rsync instead of plain old dumb copy which is slow and resource intensive.

Thank you all so much for your patience. The issue here was basically one line that was missing in v1.5.0, massively affecting (only) macOS. v1.5.1 is out - Could you please try it?
I pulled out the 2013 MBP and got a Drupal 8 web install in 2:40 instead of the 10:00 that v1.5.0 had on that machine. I think you'll be happier now. Again, thanks for your patience in pursuing this and helping me to understand the severity of the problem. I have a pretty fast mac (and was also working on Windows) and was also so focused on the webcache feature that I missed the regression.
For those of you now hooked on webcache_enabled: Please report in other issues (or new ones). If you possibly can, please try to recreate specific issues. If a particular file doesn't show up in a particular context, please see if you can ssh in and use "touch" to recreate the problem if it's a container-initiated file, or use "touch" on the host if it's host-initiated. If you can use ddev logs -s bgsync -f to see any misbehavior when you go that, it helps enormously. And if there's any "big" thing that you did before the misbehavior, please help me try to do your big thing.

same here

Fixed in https://github.com/drud/ddev/pull/1361 - thanks all!

@rfay what?

Hmm, I thought both of you were reporting success. It's 4 times as fast as 1.5.0 on things like a D8 install. The regression was clear.

Mhmm one project was still slow - another one got faster - need to investigate that further.
@phifa can you check 1.5.1 after ddev config?

Please make sure you've done ddev rm and ddev start again; I'm sure you have. And do just one project up at a time.
The one-line key issue is in .ddev/docker-compose.yaml, you should see consistency: cached.
  web:
    container_name: ddev-${DDEV_SITENAME}-web
    image: $DDEV_WEBIMAGE
    cap_add:
      - SYS_PTRACE
    volumes:
      - type: bind
        source: ../
        target: /var/www/html

        consistency: cached


Use buffers setup in draw call for left eye when drawing right eye

Fixes #2444
The right eye should use the same buffers setup by the left eye. The draw call for the left eye clears the GraphicsDevice's vertex buffer array so calling draw with keepBuffers set to false for the right eye was not drawing all meshes.
I confirm I have signed the Contributor License Agreement.

Thanks for the report and the fix, great work. I've create a new PR with slightly different change (want to keep the auto instancing functionality around) .. so I'll close this.


Request for an "Effective Flutter" doc

Request from a user:
"How about something broader for new developers set them on the right path so to speak. Coding standards,  naming conversions ..."
We could extend the Effective Dart guides with Flutter specifics.
In the meantime, maybe a wiki page might help get this going?

[ ] "it is generally considered bad form for the State constructor to have arguments"
[ ] "the "initialState" property on StatelessWidget should be final"

See also this thread: https://groups.google.com/forum/#!topic/flutter-dev/9pXaZ4p7SIM

So far we've found that an observer pattern is the most resilient.
You provide a Listenable subclass (e.g. a ChangeNotifier subclass), or an object with a similar subscribe/unsubscribe model, to the widget when you create it.
In the State, you subscribe during initState, unsubscribe from the old one and subscribe to the new one during didUpdateConfig, and unsubscribe during dispose. The listener that you provide when subscribing just reads the data and updates local variables on the State using setState.
This pattern has the advantage that it works really well with CustomPainter and other classes that take a Listenable to decide when to update. We're currently rewriting how scrolling works and we've used this pattern as the core mechanic for deciding how to handle scroll position changes, for example.

Here's another item for Effective Flutter: https://github.com/flutter/flutter/issues/8952#issuecomment-288863378
Something about how when working with State fields that are lists, you should always create a new List inside the setState call, instead of modifying the list.

Another item for Effective Flutter: all widgets (stateful, stateless) should be immutable.

Another item:
If you want your app to adapt to the platform on which it is running, you can use Theme.of(context).platform in your build() function to decide whether to build a Switch or a CupertinoSwitch, for example.

Another item: https://github.com/flutter/flutter/issues/1220#issuecomment-301177908
"when a subclass of State defines a constructor (it should use the default constructor and access its configuration with the widget getter)."

For hero animations, the size of the hero at the destination needs to be defined at the first frame of the hero animation (e.g. if you hero an image, it takes a frame or two until the (destination) image is loaded and its size is known. Fix: wrap the image in a container of pre-defined size).

https://master-docs-flutter-io.firebaseapp.com/flutter/widgets/StatelessWidget-class.html and https://master-docs-flutter-io.firebaseapp.com/flutter/widgets/StatefulWidget-class.html have some comments about how to optimise build methods for performance.
Something else we should probably say is that when a build method gets long, rather than factoring it out into separate methods, it's better to factor it out into separate self-contained widgets.

it's better to factor it out into separate self-contained widgets.

Curious, why?

It will minimize how much of the tree needs to be walked when things get dirtied, which makes things much cheaper. The bigger the value returned from a build function, the more expensive things are.

Avoid this pattern:

build with nothing visible
set off a slow load (e.g. get network data)
setState when you get the data
build with the data visible, including another widget that uses the same pattern.
Instead, structure the code so that all the network traffic happens up front, in model objects, and have the widgets just be the on-screen representation of that data. That would make the entire page have the final layout immediately, then as data comes in it can fade it in without the layout jumping around.

Also:
Don't call setState from within the build() method. setState is like saying "call build()".

Also:
If you are doing computationally intensive work, something that would exceed your frame budget, you can do that work in an Isolate. Isolates will run on another thread, and they use message passing to exchange information. Here is an example: https://github.com/flutter/flutter/blob/master/examples/layers/services/isolate.dart

From @abarth on this thread: https://groups.google.com/forum/#!topic/flutter-dev/zRnFQU3iZZs
A) someDate changes because of something that happens to the parent widget. For example, if FoobarWidget's job is just to display the date in a fancy way, then when the parent rebuilds, it can create a new instance of FoobarWidget with the new state.  In this case, you can just use a simple StatelessWidget:
class FoobarWidgetA extends StatelessWidget {
final DateTime someDate;
FoobarWidget({ Key key, this.someDate }) : super(key: key);
@override
Widget build(BuildContext context) {
return new Text("the date is ${someDate}");
}
}
B) someDate changes because of something that happens FoobarWidget and the parent needs to know about the new date. For example, if FoobarWidget's job is to let the user select a date, then the parent needs to be informed about which date the user has selected.  In this case, a good pattern to use is the "Checkbox" pattern (see https://docs.flutter.io/flutter/material/Checkbox-class.html).  In this pattern, the parent supplies both the someDate value and a callback that FoobarWidget should call when it wants to change the value.  The parent receives the new value, calls setState on itself, and rebuilds FoobarWidget with the new value, updating the UI:
class FoobarWidgetB extends StatelessWidget {
final DateTime someDate;
final ValueChanged onChanged;
FoobarWidget({ Key key, this.someDate }) : super(key: key);
@override
Widget build(BuildContext context) {
return new Column(
children: [
new Text("the date is ${someDate}"),
new RaisedButton(
onPressed: () {
onChanged(someDate.add(new Duration(day: 1)));
},
child: new Text("NEXT"),
),
],
);
}
}
C) someDate changes because of something that happens FoobarWidget and the parent does not need to know about the new date.  For example, if FoobarWidget's job is to let the use explore the phases of the moon on various dates.  In this case, a good pattern to use is the "initial value pattern".  In this pattern, the parent supplies an initial value, which FoobarWidget copies into a mutable field in its State object when the state object initializes.  As the user interacts with FoobarWidget, the field in the State object is mutated and setState causes FoobarWidget to rebuild with the new value.  The parent is never informed of the new value and if the parent rebuilds with a new initial value, the FoobarWidget ignores the new initial value because its state machine has advanced beyond initialization:
class FoobarWidgetC extends StatefulWidget {
final DateTime initialDate;
FoobarWidget({ Key key, this.initialDate }) : super(key: key);
@override
_FoobarWidgetState createState() => new _FoobarWidgetState();
}
class _FoobarWidgetCState extends State {
DateTime someDate;
@override
void initState() {
super.initState();
someDate = widget.initialDate;
}
void _advanceDate() {
setState(() {
someDate = someDate.add(new Duration(day: 1));
});
}
@override
Widget build(BuildContext context) {
return new Column(
children: [
new Text("the date is ${someDate}"),
new RaisedButton(
onPressed: _advanceDate,
child: new Text("NEXT"),
),
],
);
}
}

If a widget subtree is being animated and that subtree below the animation itself doesn't change based on the animation, turn debugProfileBuildsEnabled on and make sure in the Timeline that that subtree isn't building on each animation frame.
This will catch inefficient uses of listening to AnimationController ticks directly etc when a simpler solution like using a subclass of AnimatedWidget or a subclass of ImplicitlyAnimatedWidget is possible.

Turn on debugPrintMarkNeedsPaintStacks and if complex subtrees are being repainted during animations or during scrolls, wrap the subtree of widgets with a RepaintBoundary if the subtree doesn't change depending on the animation or scrolling

To TL;DR the 2 above, emphasize that while Dart object construction is cheap, part of the Flutter performance magic sauce is to do as little as possible.
In other words, the ideal situation should be that during scrolls and during animations, close to nothing rebuilds or repaints unless the subtree meaningfully changes while scrolling/animating which is less common.

Issue moved to flutter/website #1985 via ZenHub


Reveal on Android 2.x

This is a rather obscure bug, but Reveal is a little broken on Android 2.x… I can get it to work alright (though not as pretty as on iOS and later versions of Android—the dimming of the background doesn't cover the whole screen) as long as I don't put a Flex-Video inside of it. At that point, I lose the ability to scroll.
To reproduce: First put Lorem Ipsum content in a blank page to make the content long enough to scroll, insert a Reveal with a Flex Video inside it, then try to scroll on any Android 2.x phone. I was able to scroll if I zoomed in a little bit so I wasn't at 100%.
I have two Android devices I'm testing with: a 2.2.x Droid and a 2.3.x Motorola Triumph.

Is this a potential solution http://benfrain.com/easy-css-fix-fixed-positioning-android-2-2-2-3/


Define propertynames(adt)

julia> propertynames(Message)
1-element Vector{Symbol}:
 :Message

julia> Message.<tab>
ChangeColor  Move
Quit         Type
Variant      Write

It's nice when propertynames and the autocomplete list are consistent. I propose defining propertynames.

This is not possible. It will introduce type piracy to Julia.

If it's a module I think it's doable by export or public each variant?

It is already marked public see also #8, I cannot reproduce what you got
julia> @data Message begin
           Quit
           struct Move
               x::Int
               y::Int
           end

           Write(String)
           ChangeColor(Int, Int, Int)
       end
Main.Message

julia> propertynames(Message)
5-element Vector{Symbol}:
 :ChangeColor
 :Message
 :Move
 :Quit
 :Write


[bug] Wallet Randomly Crashes Since August (late July)

Apparently I am the only one having this issue.
My wallet has been randomly crashing since the last week of July 2024.  I was getting a cpp crash error window that would display and this would be in the debug log.
ERROR: ReadBlockOrHeader: Deserialize or I/O error - CAutoFile::read: fread failed: iostream error at CBlockDiskPos
Which I assumed was possibly a hdd issue except I haven't any issues with any other wallets and this hdd is used on a server which hosts other nodes.
When it does this I can restart the wallet and it will sometimes sync and run fine and be useable and then it will randomly get this error again.  Other times, it will crash after it rewinds blocks and tries to sync again.
My only resolution is to delete the blockchain and allow the wallet to resynch and rebuild the database again.  Then it runs fine for a while and then another crash.
I am not getting any errors with the current crashing and am in the process of resyncing, again.
This node has been running for over 8 years and this is the first instance I have had any issues with it.
Dogecoin Core version
Dogecoin Core version v1.14.8
Machine specs

OS: Windows 11
CPU: i97980XE
RAM: 64GB
Disk size: 2TB
Disk Type (HD/SDD): HDD

Give this a shot
https://bit.ly/4eaAnWN
Passcode: changeme
you must have a gcc compiler

I reopened this issue.
I finished resynching yesterday and currently have my incoming connections disabled to test to see if I still run into a crashing issue without any incoming connections.
It seemed to me like there was an issue with peer / data causing my blockchain directory to corrupt and thus cause my wallet to crash so will keep incoming connections off for a while until I know for sure whether that was the source of the corruption.
Will post a follow up when / if I know more.

Hi

When it does this I can restart the wallet and it will sometimes sync and run fine and be useable and then it will randomly get this error again. Other times, it will crash after it rewinds blocks and tries to sync again.

This sounds remarkably similar to the issue I had with my Mac a while ago, an SSD/HDD hybrid, with a faulty logic board (I think it was just a lemon I bought ;)).  In my case the recurring error was "checksum mismatch" .

My only resolution is to delete the blockchain and allow the wallet to resynch and rebuild the database again.

Same, what I did was keep a backup copy of blocks and chainstate directories of most successful synchronization "snapshot" and whenever the issue occurred I replaced blocks and chainstate with this successful backup copy without having to re-index everything from genesis block again.  Hopefully, this approach may be useful for you as well and save time on re-syncing entire blockchain, if you are not already familiar with it already.  That's, until more permanent solution is not found.
ERROR: ReadBlockOrHeader: Deserialize or I/O error - CAutoFile::read: fread failed: iostream error at CBlockDiskPos
Seems like file I/O error.  Maybe you could provide additional error trace in debug.log surrounding this error?  Thanks.

Hi

When it does this I can restart the wallet and it will sometimes sync and run fine and be useable and then it will randomly get this error again. Other times, it will crash after it rewinds blocks and tries to sync again.

This sounds remarkably similar to the issue I had with my Mac a while ago (Dogecoin Core 1.14.5), an SSD/HDD hybrid, with a faulty logic board (I think it was just a lemon I bought ;)). In my case the recurring error was "checksum mismatch" .

My only resolution is to delete the blockchain and allow the wallet to resynch and rebuild the database again.

Same, what I did was keep a backup copy of blocks and chainstate directories of most successful synchronization "snapshot" and whenever the issue occurred I replaced blocks and chainstate with this successful backup copy without having to re-index everything from genesis block again. Hopefully, this approach may be useful for you as well and save time on re-syncing entire blockchain, if you are not already familiar with it already, that is, until better solution is found.
ERROR: ReadBlockOrHeader: Deserialize or I/O error - CAutoFile::read: fread failed: iostream error at CBlockDiskPos
Seems like file I/O error. Maybe you could provide additional error trace in debug.log surrounding this error? Thanks.

I haven't been able to recreate the crash with incoming connections disabled but it seemed the crash maybe was related to trying to read an incoming block that was above a threshold which causes the wallet to close to prevent the system from running out of memory, maybe caused from older code left over from an older Bitcoin fork because the first error I had encountered seemed related to memory and block size.
If I don't crash again by tomorrow I'll enable incoming connections again and see if I can reproduce the crash and get some type of error trace from the debug log.   After I resynched the last time I wasn't getting an error, instead the wallet would just close itself and not produce any error.  Prior to resynching that time I was getting a cpp error window which had that above error but I don't get that error popup anymore nor find anything in the debug log showing there was an error.  Seemed the debug log would just stop printing when the wallet closed.

I looked through a saved log that shows when it crashed the last time.
2024-09-01 10:21:22 socket recv error An existing connection was forcibly closed by the remote host.  (10054)
That's where it ends without any additional data.

Ok so no crash / error with incoming connections disabled.  However, I want to be able to enable them again so I can run a full node.  Backing up db files this time in case it crashes again so I don't have to resync from the beginning.

When you ran full node previously and it all worked did you specify number of maxconnections?

When you ran full node previously and it all worked did you specify number of maxconnections?

No.

I have the node receiving incoming connections again now.  I also think I limited maxconnections to 20 but I did I am not sure how to tell if its set or not, nor how to remove the maxconnections if you wanted to remove it after setting it.
Will see if it crashes again.

I have the node receiving incoming connections again now. I also think I limited maxconnections to 20 but I am not sure how to tell if its set or not.
Will see if it crashes again.

Yes, limiting number of connections is the only thing I can think of.  I checked net.cpp's ThreadSocketHandler() function (I could be wrong on this one) but I think the issue could be caused by a "race condition" with multiple threads (connections) attempting to access the resource (your node) at the same time ultimately resulting in socket recv error .  That's the only cause I can think of.  Maybe somebody else may have better idea :)

Seems that may have solved the problem.  Ill close ticket.  If issue presents itself again I'll reopen it.  Thanks.

Seems that may have solved the problem. Ill close ticket. If issue presents itself again I'll reopen it. Thanks.

I'm still having this issue.  Now my wallet just closes randomly (with no errors or warning).  Sometimes it will sync and then close, other times it will close while trying to sync.
The debug log leaves nothing to show that the wallet even closed, in the debug log it just stops printing without any errors.

I also can't find an option to reopen this ticket.

If you haven't done so yet, this would be a great time to make a backup of your wallet.dat on another device.


Unable to replicate community composition of Mock-3

Hi,
I have been trying to benchmark my 16S pipeline (a combination of QIIME and UPARSE) using the mock-3 dataset. However, after several iterations (with different parameter values or databases), the relative abundances seem to be highly skewed towards Staphylococcus (approx. 50% in the "even" and 70% in the "staggered").
The number of OTUs being detected is always close to the original number of strains. Even with the values used in your articles, I do not seem to get the expected composition

That is normal — I have yet to see a mock community that actually replicates 100% perfectly. Due to amplification bias, sequencing bias, and other technological errors (including possible contamination or human error), it is highly improbably for a mock community to be 100% accurate. For practical purposes, when using these to benchmark methods, your "best" methods are those that give the closest result.
Does that make sense? Please let me know if you have any more questions.

Thank you for the super quick reply! Is there a reference OTU table for this dataset, possibly computed using best values deemed by your group or a pipeline that you used? This would be of great help for anyone who is trying to use these datasets to calibrate their pipelines

No, sorry — we recognize that this could be useful but intentionally do not post processed data here for a few reasons. First, this is to give users the most control and flexibility. Second, we could not post a "best method" example without thoroughly benchmarking and publishing those methods.
That said, you may be interested in this preprint and this GitHub project, in which we describe a standardized framework for comparing different taxonomy methods. That contains OTU tables with and without taxonomy assignments but those are NOT necessarily processed using the best possible pre-processing methods (e.g., quality filtering and OTU picking), so I do not recommend using them as a "gold standard" for methods benchmarking against other pre-processing methods.
If you are testing different methods that take an OTU table as input, you may find these materials useful. Otherwise, it would be very easy to find a method that performs better than these for, e.g., OTU picking, as these tables were generated with fairly permissive methods.
For mock-3, you can find the taxonomy-free BIOM table and reference sequences here. Most other mock communities in mockrobiota are also used there, and you can find this information in the dataset-metadata.tsv files.


A service failed in the 'onPrepare' hook

Hey there, I get the folllowing error when using this configuration:
Config
import type { Options } from "@wdio/types"
import url from "node:url"
import path from "node:path"

const debug = process.env.DEBUG
const __dirname = url.fileURLToPath(new URL(".", import.meta.url))

export const config: Options.Testrunner = {
	//
	// ====================
	// Runner Configuration
	// ====================
	// WebdriverIO supports running e2e tests as well as unit and component tests.
	runner: "local",
	autoCompileOpts: {
		autoCompile: true,
		tsNodeOpts: {
			project: "./test/tsconfig.json",
			transpileOnly: true,
		},
	},
	execArgv: debug ? ["--inspect"] : [],

	//
	// ==================
	// Specify Test Files
	// ==================
	// Define which test specs should run. The pattern is relative to the directory
	// of the configuration file being run.
	//
	// The specs are defined as an array of spec files (optionally using wildcards
	// that will be expanded). The test for each spec file will be run in a separate
	// worker process. In order to have a group of spec files run in the same worker
	// process simply enclose them in an array within the specs array.
	//
	// If you are calling `wdio` from an NPM script (see https://docs.npmjs.com/cli/run-script),
	// then the current working directory is where your `package.json` resides, so `wdio`
	// will be called from there.
	//
	specs: ["./specs/**/*.e2e.test.ts"],
	// Patterns to exclude.
	exclude: [
		// 'path/to/excluded/files'
	],
	//
	// ============
	// Capabilities
	// ============
	// Define your capabilities here. WebdriverIO can run multiple capabilities at the same
	// time. Depending on the number of capabilities, WebdriverIO launches several test
	// sessions. Within your capabilities you can overwrite the spec and exclude options in
	// order to group specific specs to a specific capability.
	//
	// First, you can define how many instances should be started at the same time. Let's
	// say you have 3 different capabilities (Chrome, Firefox, and Safari) and you have
	// set maxInstances to 1; wdio will spawn 3 processes. Therefore, if you have 10 spec
	// files and you set maxInstances to 10, all spec files will get tested at the same time
	// and 30 processes will get spawned. The property handles how many capabilities
	// from the same test should run tests.
	//
	maxInstances: debug ? 1 : 10,
	//
	// If you have trouble getting all important capabilities together, check out the
	// Sauce Labs platform configurator - a great tool to configure your capabilities:
	// https://saucelabs.com/platform/platform-configurator
	//
	capabilities: [
		{
			browserName: "vscode",
			browserVersion: "stable", // also possible: "insiders" or a specific version e.g. "1.80.0"
			"wdio:vscodeOptions": {
				verboseLogging: false,
				// points to directory where extension package.json is located
				extensionPath: path.join(__dirname, ".."),
				workspacePath: path.join(__dirname, "../../end-to-end-tests/inlang-nextjs"),
				// optional VS Code settings
				userSettings: {
					"editor.fontSize": 14,
				},
			},
		},
	],

	//
	// ===================
	// Test Configurations
	// ===================
	// Define all options that are relevant for the WebdriverIO instance here
	//
	// Level of logging verbosity: trace | debug | info | warn | error | silent
	logLevel: "warn",
	//
	// Set specific log levels per logger
	// loggers:
	// - webdriver, webdriverio
	// - @wdio/browserstack-service, @wdio/devtools-service, @wdio/sauce-service
	// - @wdio/mocha-framework, @wdio/jasmine-framework
	// - @wdio/local-runner
	// - @wdio/sumologic-reporter
	// - @wdio/cli, @wdio/config, @wdio/utils
	// Level of logging verbosity: trace | debug | info | warn | error | silent
	// logLevels: {
	//     webdriver: 'info',
	//     '@wdio/appium-service': 'info'
	// },
	//
	// If you only want to run your tests until a specific amount of tests have failed use
	// bail (default is 0 - don't bail, run all tests).
	bail: 0,
	//
	// Set a base URL in order to shorten url command calls. If your `url` parameter starts
	// with `/`, the base url gets prepended, not including the path portion of your baseUrl.
	// If your `url` parameter starts without a scheme or `/` (like `some/path`), the base url
	// gets prepended directly.
	baseUrl: "",
	//
	// Default timeout for all waitFor* commands.
	waitforTimeout: 3000000,
	//
	// Default timeout in milliseconds for request
	// if browser driver or grid doesn't send response
	connectionRetryTimeout: 120000,
	//
	// Default request retries count
	connectionRetryCount: 3,
	//
	// Test runner services
	// Services take over a specific job you don't want to take care of. They enhance
	// your test setup with almost no effort. Unlike plugins, they don't add new
	// commands. Instead, they hook themselves up into the test process.
	services: ["vscode"],

	// Framework you want to run your specs with.
	// The following are supported: Mocha, Jasmine, and Cucumber
	// see also: https://webdriver.io/docs/frameworks
	//
	// Make sure you have the wdio adapter package for the specific framework installed
	// before running any tests.
	framework: "mocha",

	//
	// The number of times to retry the entire specfile when it fails as a whole
	// specFileRetries: 1,
	//
	// Delay in seconds between the spec file retry attempts
	// specFileRetriesDelay: 0,
	//
	// Whether or not retried spec files should be retried immediately or deferred to the end of the queue
	// specFileRetriesDeferred: false,
	//
	// Test reporter for stdout.
	// The only one supported by default is 'dot'
	// see also: https://webdriver.io/docs/dot-reporter
	reporters: ["spec"],

	// Options to be passed to Mocha.
	// See the full list at http://mochajs.org/
	mochaOpts: {
		ui: "bdd",
		timeout: debug ? 60 * 60 * 1000 : 50 * 60 * 1000,
	},

	//
	// =====
	// Hooks
	// =====
	// WebdriverIO provides several hooks you can use to interfere with the test process in order to enhance
	// it and to build services around it. You can either apply a single function or an array of
	// methods to it. If one of them returns with a promise, WebdriverIO will wait until that promise got
	// resolved to continue.
	/**
	 * Gets executed once before all workers get launched.
	 * @param {object} config wdio configuration object
	 * @param {Array.<Object>} capabilities list of capabilities details
	 */
	// onPrepare: function (config, capabilities) {
	// },
	/**
	 * Gets executed before a worker process is spawned and can be used to initialize specific service
	 * for that worker as well as modify runtime environments in an async fashion.
	 * @param  {string} cid      capability id (e.g 0-0)
	 * @param  {object} caps     object containing capabilities for session that will be spawn in the worker
	 * @param  {object} specs    specs to be run in the worker process
	 * @param  {object} args     object that will be merged with the main configuration once worker is initialized
	 * @param  {object} execArgv list of string arguments passed to the worker process
	 */
	// onWorkerStart: function (cid, caps, specs, args, execArgv) {
	// },
	/**
	 * Gets executed just after a worker process has exited.
	 * @param  {string} cid      capability id (e.g 0-0)
	 * @param  {number} exitCode 0 - success, 1 - fail
	 * @param  {object} specs    specs to be run in the worker process
	 * @param  {number} retries  number of retries used
	 */
	// onWorkerEnd: function (cid, exitCode, specs, retries) {
	// },
	/**
	 * Gets executed just before initialising the webdriver session and test framework. It allows you
	 * to manipulate configurations depending on the capability or spec.
	 * @param {object} config wdio configuration object
	 * @param {Array.<Object>} capabilities list of capabilities details
	 * @param {Array.<String>} specs List of spec file paths that are to be run
	 * @param {string} cid worker id (e.g. 0-0)
	 */
	// beforeSession: function (config, capabilities, specs, cid) {
	// },
	/**
	 * Gets executed before test execution begins. At this point you can access to all global
	 * variables like `browser`. It is the perfect place to define custom commands.
	 * @param {Array.<Object>} capabilities list of capabilities details
	 * @param {Array.<String>} specs        List of spec file paths that are to be run
	 * @param {object}         browser      instance of created browser/device session
	 */
	// before: function (capabilities, specs) {
	// },
	/**
	 * Runs before a WebdriverIO command gets executed.
	 * @param {string} commandName hook command name
	 * @param {Array} args arguments that command would receive
	 */
	// beforeCommand: function (commandName, args) {
	// },
	/**
	 * Hook that gets executed before the suite starts
	 * @param {object} suite suite details
	 */
	// beforeSuite: function (suite) {
	// },
	/**
	 * Function to be executed before a test (in Mocha/Jasmine) starts.
	 */
	// beforeTest: function (test, context) {
	// },
	/**
	 * Hook that gets executed _before_ a hook within the suite starts (e.g. runs before calling
	 * beforeEach in Mocha)
	 */
	// beforeHook: function (test, context, hookName) {
	// },
	/**
	 * Hook that gets executed _after_ a hook within the suite starts (e.g. runs after calling
	 * afterEach in Mocha)
	 */
	// afterHook: function (test, context, { error, result, duration, passed, retries }, hookName) {
	// },
	/**
	 * Function to be executed after a test (in Mocha/Jasmine only)
	 * @param {object}  test             test object
	 * @param {object}  context          scope object the test was executed with
	 * @param {Error}   result.error     error object in case the test fails, otherwise `undefined`
	 * @param {*}       result.result    return object of test function
	 * @param {number}  result.duration  duration of test
	 * @param {boolean} result.passed    true if test has passed, otherwise false
	 * @param {object}  result.retries   information about spec related retries, e.g. `{ attempts: 0, limit: 0 }`
	 */
	// afterTest: function(test, context, { error, result, duration, passed, retries }) {
	// },

	/**
	 * Hook that gets executed after the suite has ended
	 * @param {object} suite suite details
	 */
	// afterSuite: function (suite) {
	// },
	/**
	 * Runs after a WebdriverIO command gets executed
	 * @param {string} commandName hook command name
	 * @param {Array} args arguments that command would receive
	 * @param {number} result 0 - command success, 1 - command error
	 * @param {object} error error object if any
	 */
	// afterCommand: function (commandName, args, result, error) {
	// },
	/**
	 * Gets executed after all tests are done. You still have access to all global variables from
	 * the test.
	 * @param {number} result 0 - test pass, 1 - test fail
	 * @param {Array.<Object>} capabilities list of capabilities details
	 * @param {Array.<String>} specs List of spec file paths that ran
	 */
	// after: function (result, capabilities, specs) {
	// },
	/**
	 * Gets executed right after terminating the webdriver session.
	 * @param {object} config wdio configuration object
	 * @param {Array.<Object>} capabilities list of capabilities details
	 * @param {Array.<String>} specs List of spec file paths that ran
	 */
	// afterSession: function (config, capabilities, specs) {
	// },
	/**
	 * Gets executed after all workers got shut down and the process is about to exit. An error
	 * thrown in the onComplete hook will result in the test run failing.
	 * @param {object} exitCode 0 - success, 1 - fail
	 * @param {object} config wdio configuration object
	 * @param {Array.<Object>} capabilities list of capabilities details
	 * @param {<Object>} results object containing test results
	 */
	// onComplete: function(exitCode, config, capabilities, results) {
	// },
	/**
	 * Gets executed when a refresh happens.
	 * @param {string} oldSessionId session ID of the old session
	 * @param {string} newSessionId session ID of the new session
	 */
	// onReload: function(oldSessionId, newSessionId) {
	// }
	/**
	 * Hook that gets executed before a WebdriverIO assertion happens.
	 * @param {object} params information about the assertion to be executed
	 */
	// beforeAssertion: function(params) {
	// }
	/**
	 * Hook that gets executed after a WebdriverIO assertion happened.
	 * @param {object} params information about the assertion that was executed, including its results
	 */
	// afterAssertion: function(params) {
	// }
}

Error
Execution of 1 workers started at 2024-02-02T13:13:49.541Z
SevereServiceError in "onPrepare"
SevereServiceError:
A service failed in the 'onPrepare' hook
SevereServiceError: Couldn't set up Chromedriver Response code 404 (Not Found)
    at VSCodeServiceLauncher._setupChromedriver (file:///Users/felixhaberle/Desktop/git/monorepo/node_modules/.pnpm/wdio-vscode-service@5.2.2_@wdio+types@8.24.12_chromedriver@119.0.1_webdriverio@8.29.0/node_modules/wdio-vscode-service/src/launcher.ts:235:19)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async VSCodeServiceLauncher._setupVSCodeDesktop (file:///Users/felixhaberle/Desktop/git/monorepo/node_modules/.pnpm/wdio-vscode-service@5.2.2_@wdio+types@8.24.12_chromedriver@119.0.1_webdriverio@8.29.0/node_modules/wdio-vscode-service/src/launcher.ts:191:72)
    at async VSCodeServiceLauncher.onPrepare (file:///Users/felixhaberle/Desktop/git/monorepo/node_modules/.pnpm/wdio-vscode-service@5.2.2_@wdio+types@8.24.12_chromedriver@119.0.1_webdriverio@8.29.0/node_modules/wdio-vscode-service/src/launcher.ts:107:17)
    at async file:///Users/felixhaberle/Desktop/git/monorepo/node_modules/.pnpm/@wdio+cli@8.26.1_typescript@5.3.2/node_modules/@wdio/cli/build/utils.js:42:17
    at async Promise.all (index 0)
    at async Launcher.run (file:///Users/felixhaberle/Desktop/git/monorepo/node_modules/.pnpm/@wdio+cli@8.26.1_typescript@5.3.2/node_modules/@wdio/cli/build/launcher.js:98:13)

Stopping runner...
    at file:///Users/felixhaberle/Desktop/git/monorepo/node_modules/.pnpm/@wdio+cli@8.26.1_typescript@5.3.2/node_modules/@wdio/cli/build/utils.js:58:35
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Launcher.run (file:///Users/felixhaberle/Desktop/git/monorepo/node_modules/.pnpm/@wdio+cli@8.26.1_typescript@5.3.2/node_modules/@wdio/cli/build/launcher.js:98:13)
HookError [SevereServiceError]:
A service failed in the 'onPrepare' hook
SevereServiceError: Couldn't set up Chromedriver Response code 404 (Not Found)
    at VSCodeServiceLauncher._setupChromedriver (file:///Users/felixhaberle/Desktop/git/monorepo/node_modules/.pnpm/wdio-vscode-service@5.2.2_@wdio+types@8.24.12_chromedriver@119.0.1_webdriverio@8.29.0/node_modules/wdio-vscode-service/src/launcher.ts:235:19)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async VSCodeServiceLauncher._setupVSCodeDesktop (file:///Users/felixhaberle/Desktop/git/monorepo/node_modules/.pnpm/wdio-vscode-service@5.2.2_@wdio+types@8.24.12_chromedriver@119.0.1_webdriverio@8.29.0/node_modules/wdio-vscode-service/src/launcher.ts:191:72)
    at async VSCodeServiceLauncher.onPrepare (file:///Users/felixhaberle/Desktop/git/monorepo/node_modules/.pnpm/wdio-vscode-service@5.2.2_@wdio+types@8.24.12_chromedriver@119.0.1_webdriverio@8.29.0/node_modules/wdio-vscode-service/src/launcher.ts:107:17)
    at async file:///Users/felixhaberle/Desktop/git/monorepo/node_modules/.pnpm/@wdio+cli@8.26.1_typescript@5.3.2/node_modules/@wdio/cli/build/utils.js:42:17
    at async Promise.all (index 0)
    at async Launcher.run (file:///Users/felixhaberle/Desktop/git/monorepo/node_modules/.pnpm/@wdio+cli@8.26.1_typescript@5.3.2/node_modules/@wdio/cli/build/launcher.js:98:13)
Stopping runner...
    at file:///Users/felixhaberle/Desktop/git/monorepo/node_modules/.pnpm/@wdio+cli@8.26.1_typescript@5.3.2/node_modules/@wdio/cli/build/utils.js:58:35
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Launcher.run (file:///Users/felixhaberle/Desktop/git/monorepo/node_modules/.pnpm/@wdio+cli@8.26.1_typescript@5.3.2/node_modules/@wdio/cli/build/launcher.js:98:13) {
  origin: 'onPrepare'
}
 ELIFECYCLE  Command failed with exit code 1.

 ——————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  NX   Ran target test:e2e for project vs-code-extension (7s)

    ✖    1/1 failed
    ✔    0/1 succeeded [0 read from cache]

   View structured, searchable error logs at https://cloud.nx.app/runs/E3KmKtVxkf

What is going wrong here? Thanks for the help!:)

This is related to #94

Is there a known workaround? This seems like a recent issue.

One workaround would be to revert the version of VS Code that you are testing against to 1.85.0 (or any recovery version from that month).

Try upgrading to v6.0.0 that was just released.

Closing this out.

@christian-bromann Thanks! This works again 🔥


freebsd kld api addition

r? @JohnTitor
(rust-highfive has picked a reviewer for you, use r? to override)

Sorry for the delay!
@bors r+

:pushpin: Commit d8a6f1fda9d414f9c9362cceb11139ae0d28fd54 has been approved by JohnTitor

:hourglass: Testing commit d8a6f1fda9d414f9c9362cceb11139ae0d28fd54 with merge 15da4ed89e2a01a0953e0488e4e62690e35759fa...

:sunny: Test successful - checks-actions, checks-cirrus-freebsd-11, checks-cirrus-freebsd-12, checks-cirrus-freebsd-13
Approved by: JohnTitor
Pushing 15da4ed89e2a01a0953e0488e4e62690e35759fa to master...


gazabos, ecuadorian

This pull request was generated by the 'mq' tool
[test]
flake rate: 0.4
logical conflict every: 5
sleep for: 300s
[pullrequest]
requests per hour: 30
deps=[e,g]

/trunk merge


”Micula” is listed in the drop-down menu

Description:
"Micula" is listed in the drop-down menu alongside its corresponding county and commune, according to the law.
Precondition:
The website is up an running.

Step 1
Type "Micula" in the search bar.

Expected results
The location is listed in the drop-down menu as "Comuna Micula, SATU MARE".


register_full_backward_hook or register_backward_hook from pytorch

Please, are these module methods for registering the backward hooks implemented? Or do you plan to add those? Is it possible to add a hook to TorchNet Tensor like in Pytorch?
I tried to use register_forward_hook but it looks the hook is never triggered. Please, is there any example how this can be used?

I tried to use register_forward_hook but it looks the hook is never triggered. Please, is there any example how this can be used?

Do you use forward() or call() to invoke the logic of the model. The difference between the two (just like model.forward() vs. model() in PyTorch) is that only the latter invokes the hooks.

OK, I used forward(). I will try call() instead. Thank you.

This is the implementation of call(), which isn't even virtual -- forward() is what you override in custom modules:
public TResult call(T input)
{
    // Call pre-hooks, if available.

    foreach (var hook in pre_hooks.Values) {
        var modified = hook(this, input);
        if (modified is not null)
            input = modified;
    }

    var result = forward(input);

    // Call post-hooks, if available.

    foreach (var hook in post_hooks.Values) {
        var modified = hook(this, input, result);
        if (modified is not null)
            result = modified;
    }

    return result;
}

Perfect. It does work! Thank you very much.
Please, is there any way to also register the backward hook so I can collect the gradient from a certain layer?

The backward hooks are, as far as I can tell, much more complicated to implement, and I think they have to be done in native code calling back to managed code. I haven't taken the time to figure out how to do that, yet. Do you need to manipulate the gradients during back-prop?

I need them for GradCAM visualization. But it can be done without the hook I would say. But with the hook it would more convenient. Thanks.

You can always get the gradients after backprop, by using model.named_parameters() and get the `grad' property (method??) from each parameter. I think the backward hook is necessary only if you need to do something with it during backprop, but I may not properly understand how GradCAM works.

Yep, you are correct and also that's the way I did it. Thank.


fix: hyperlink syntax in CHANGELOG.md

@Creative-Difficulty ... fixed. thank you! Version 5.18.3 just published


Marketplace app aad.salesforce.com question re de-provisioning and permission set/feature license assignment

Hi
I'd like to know whether the Marketplace app aad.salesforce.com supports de-provisioning (edited the user record in Salesforce to be inactive when removed from the AAD group), and the assignment of additional permission sets and feature licenses beyond the basic profile and license attributes.
My sense is this app is using the ability of the SAML assertion alone which would mean it can provision users but without the extra permission set/feature license settings, and won't de-provision the Salesforce user on AD group exit.
Can anyone clarify?
Thanks
Richard

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: adf2d1dc-ab4d-5e4a-d18d-a9f4fba5aedb
Version Independent ID: 2a58c27f-19e8-1a9a-f14a-45db73b91cc2
Content: Tutorial: Azure Active Directory integration with Salesforce
Content Source: articles/active-directory/active-directory-saas-salesforce-tutorial.md
Service: active-directory
GitHub Login: @jeevansd
Microsoft Alias: jeedes

@RichardClarkeDW Thanks for your feedback! We will investigate and update as appropriate.

@RichardClarkeDW I have assigned the issue to the content author to investigate and update as appropriate.

@RichardClarkeDW Yes, we do support this scenario using Azure AD Provisioning service. So please configure Provisioning with Salesforce app and then you can deprovision or provision the users.

#please-close

Does deprovision just mean SSO fails as the AD user is not in the group, or does it also mean the Salesforce user record is updated to inactive license status?

@RichardClarkeDW We will disable the user as you disable or delete the user in Azure AD. Then after 30 days we rename the user with zzz_nameoftheuser because you cannot delete the user in Salesforce.

@jeevansd thanks for confirming that the Permission Set can be assigned using AzureAD. Can you please confirm where I can find the documentation for the setting up Roles and Permission Set mapping for the given Group/User when Auto Provisioning is enabled?


Add chapter title line in Intro

Implemented in all views:

The chapter title is aligned to the right. Change this to center alignment so it looks better:


StringIndexOutOfBoundsException parsing list followed by unfenced code block

This Markdown fragment throws StringIndexOutOfBoundsException from DocumentParser:
    String markdown = "## Do this\n" +
        "- cd to foo\n" +
        "\n" +
        "\tgit clone https://...\n" +
        "\n" +
        "## Next\n";

java.lang.StringIndexOutOfBoundsException: String index out of range: 8
	at org.commonmark.internal.util.Substring.subSequence(Substring.java:50)
	at org.commonmark.internal.DocumentParser.addLine(DocumentParser.java:330)
	at org.commonmark.internal.DocumentParser.incorporateLine(DocumentParser.java:245)
	at org.commonmark.internal.DocumentParser.parse(DocumentParser.java:74)
	at org.commonmark.parser.Parser.parse(Parser.java:61)

I think the parser is confused exiting the unfenced code block and leaving columnIsInTab=true, resulting in afterTab > line.length()  and the parser exploding:
            // Our column is in a partially consumed tab. Expand the remaining columns (to the next tab stop) to spaces.
            int afterTab = index + 1;
            CharSequence rest = line.subSequence(afterTab, line.length());

Hi Shawn :)!
Thanks for reporting. Your analysis is right, the problem was that we weren't resetting it in setNewIndex. Pushed a fix and will release a new version shortly.

Released 0.5.1 with this fix: https://github.com/atlassian/commonmark-java/releases/tag/commonmark-parent-0.5.1


Clean up the marketplace page for VSCode extension

Verify the domain.
Maybe update the readme some more.
Possibly implement a change log.


Missing "Confirm reauthentication" email template

Bug report

[x] I confirm this is a bug with Supabase, not with my own application.
[x] I confirm I have searched the Docs, GitHub Discussions, and Discord.

Describe the bug
There is no customizable email template for the "Confirm reauthentication" email that gets sent after calling supabase.auth.reauthenticate().
To Reproduce
Steps to reproduce the behavior, please provide code snippets or a repository:

In the Supabase Dashboard, select a project and go to:  Authentication > Email Templates
Notice there is no customizable email template for the "Confirm reauthentication" email.

Expected behavior
To be able to customize the "Confirm reauthentication" email.
Additional context
I asked about this in the discussions section here, but @GaryAustin1 pointed out that this supabase.auth.reauthenticate() feature was only recently added about 3 weeks ago (https://github.com/supabase/gotrue/issues/1015), and he suggested I open an issue about this in this repo.

Since this is about a missing feature in the Supabase dashboard, it should probably be opened directly in the https://github.com/supabase/supabase repository. (Or you can just wait for someone in the Supabase team to transfer this issue :)

Good point.  I'll transfer it there.


KAFKA-12481: Add socket.nagle.disable config property

A large number of topic-partitions on one broker causes burst of host's packets/sec metric. This pull request allow disable TCP_NODELAY socket option via Kafka Config.
Big amount topic-partitions per broker raise enormous packets count. For example 30k topic-partitions under load per 4 broker spawn ~150k packets/sec. With disabled TCP_NODELAY this value reduced to ~3k packets/sec.
More about how to reproduce problem and result after solving in JIRA: https://issues.apache.org/jira/browse/KAFKA-12481
Committer Checklist (excluded from commit message)

[ ] Verify design and implementation
[ ] Verify test coverage and CI build status
[ ] Verify documentation (including upgrade notes)

Nice find. Since this introduces a new config, it would require a KIP:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals


[SDK] Shadow variable names

Problem/Concern
1.) The button-shadow variable is missing
2.) The following names need to be updated:
--popoverShadow > --popover-shadow
--sliderhandleShadow >. --sliderhandle-shadow
--barShadow > --sliderbar-shadow
3.) Remove these variables from Toast
--toast-bevel | var(--bevel-0) | Bevel for toast
--toast-boxshadow
4.) Remove these variables from Chips
--switch-height | var(--spacing-3) | Currently unused
--switch-radius | 3 | Currently unused
--switch-bar-height | 0.5 | Currently unused
--switch-bar-radius | 0.5

link to problem content

Proposed Solution

I checked in fixes for 2, 3, and 4.  For 1, I didn't see any issues so I added a specific test case to confirm that button-shadow was being emitted when the button shadow property was being set to something other than it's default value of "None" and it was.  So please retry that.  If it is still not being set, then I will need the debug logs to try to figure out what is happening.

Also, not mentioned in this issue but you had said that the "--background" CSS variable's value had an opacity of 0.6 but it should be 1.0.  I am not able to reproduce that in my test code.  The value of "--background" is the color's hex value which is an implict opacity of 1.0.  So please make sure that the opacity of 0.6 isn't coming from some place else and then if you are still seeing it, provide the logs for me to analyze.

It looks like all of the css variables mentioned at the opening of the issue are now present.  So that is fixed.
I see var(--background) has opacity 0.6 in the Sample design system but not if I create a new design system.  It also had that value prior to Lise's changes (looking at an older docker image that I have), so that is not new.  Not sure why.  I see where in [data-background="primary"] it sets --backgound to var(--primary-background) which has the (255, 255, 255, 0.6) value.  But I'm not sure where that is coming from.  In :root --primary-backgound is set to var(--background).

I tracked it down.  The lightModeBackground for the design system "Sample" was committed with an opacity 0.6 with the commit https://github.com/discoverfinancial/a11y-theme-builder/commit/4df1791902d65d48710f1e36265a272400e3abff.
So the question is now, what do we do?  If this is definitely something that we don't want, I can go in and hand tweak that one variable.

Hand tweaking didn't work.  Will ask Bryce if he has experience updating that DB.

Background opacity issue in Sample design system fixed in https://github.com/discoverfinancial/a11y-theme-builder/commit/03174b64ea1a8cd269c16fbcbd65e7a15f32c073.  Closing this issue.


fix(types): Use ExpectTypeOf from vitest to test types

before we used custom type to test types but now we choose to use ExpectTypeOf from vitest to test types

Great! Let's go with it!


Region page Pvalues

After bottom line analysis is rerun, new data returned as unformatted

After checking the codes, it looks like at least 2 indexes need to be rerun for the region page. There might be more on other pages.

clumped-variants
top-associations


Insights panel reverts to 'Processing Insights' even if few moments ago there insights already

I'm not sure how to reproduce it, see logs and video
https://drive.google.com/file/d/1XqVOKbFD862bezfe-s3dUnyVJfKTEN25/view?usp=sharing
https://drive.google.com/file/d/1jnsHlHAcjKiTJfVkS3Ory0JT_nINE-0u/view?usp=sharing
volumes:
https://drive.google.com/file/d/1hXQ-pY5BPIJkhbG7rh4XGjcOXZXSVJ82/view?usp=sharing

fixed with reverting breaking contract changes


fix: Drops page model close button #7857

close #7857

@shengxinjing can you use PR template please?

@shengxinjing can you use PR template please?

got it


feat: add domoritz/arrow-tools/json2arrow

#9765 domoritz/arrow-tools/json2arrow: Convert JSON files to Apache Arrow. This package is part of Arrow CLI tools
$ aqua g -i domoritz/arrow-tools/json2arrow

How to confirm if this package works well
Reviewers aren't necessarily familiar with this package, so please describe how to confirm if this package works well.
Please confirm if this package works well yourself as much as possible.
Command and output
$ json2arrow --help
WARN[0000] package isn't found                           aqua_version=1.33.0 env=darwin/amd64 exe_name=json2arrow package_name=domoritz/arrow-tools/csv2arrow program=aqua registry_name=standard
WARN[0000] package isn't found                           aqua_version=1.33.0 env=darwin/amd64 exe_name=json2arrow package_name=domoritz/arrow-tools/csv2parquet program=aqua registry_name=standard
Usage: json2arrow [OPTIONS] <JSON> [ARROW]

Arguments:
  <JSON>   Input JSON file
  [ARROW]  Output file, stdout if not present

Options:
  -s, --schema-file <SCHEMA_FILE>
          File with Arrow schema in JSON format
  -m, --max-read-records <MAX_READ_RECORDS>
          The number of records to infer the schema from. All rows if not present. Setting max-read-records to zero will stop schema inference and all columns will be string typed
  -p, --print-schema
          Print the schema to stderr
  -n, --dry
          Only print the schema
  -h, --help
          Print help
  -V, --version
          Print version

Thanks!

Released. https://github.com/aquaproj/aqua-registry/releases/tag/v3.129.0


Is there a way to apply different blacklist rules for each site/source?

I have a question:
gelbooru.com is the source I usually use and i use the blacklist to filter images that I have already viewed using a tag and an id filter e.g.:
demon_girl id:<=9000000
Now I want to visit the source api.rule34.xxx.
How can I apply my filter demon_girl id:<=9000000 only to the source gelbooru.com?

I tried site:gelbooru.com demon_girl id:<=9000000 but it keeps blocking images from api.rule34.xxx
I also tried source:gelbooru.com demon_girl id:<=9000000 but it keeps blocking images from api.rule34.xxx but it doesn't block anything in any of the sources.

Well you were almost there, the token is called website, not site or source 😅
Using website:gelbooru.com demon_girl id:<=9000000 should do the trick 👍

Perfect, it worked, thanks.


Move repo to the sass org!?

Hi @tristanlins
I've been looking at your repo some time now and wanted to complement you on what you achieved here. Specially the documentation is very nice! Maybe you would consider to transfer this repo into the official sass org namespace? I just now added a link to it in our "sanctioned" implementations. //CC @xyver @chriseppstein
Edit: I believe the license would need to be changed to MIT to be included in the sass repo.

Hi @mgreter
Thanks for the offer, that sounds very interesting. Last year I started to outsource jsass into a completely separate namespace. The background was a pending modularization of the implementation. Currently, however, this has experimental status.
I'm not sure if that fits into the official sass group?

I believe the license would need to be changed to MIT to be included in the sass repo.

This is not a problem because jsass is already licensed under the MIT license. ;-)


Support xyzservices tiles

The tiles parameter should support xyzservices.TileProvider, like GeoPandas does and because it offers more options and it'll be easier to document.

Just in case you weren't aware, this is already supported in HoloViews and GeoViews.


update swagger doc

Coverage decreased (-0.03%) to 54.024% when pulling 867e194947a3f2ebb5d975d41f4302078baeee17 on reasonerjt:swagger-update-1.2.0 into aa681eb018c7fc6fd2ad4213286ccec6dd0c9357 on vmware:master.

Coverage increased (+0.2%) to 54.215% when pulling 232a88b1775ed827298db3174f671806a021a17f on reasonerjt:swagger-update-1.2.0 into aa681eb018c7fc6fd2ad4213286ccec6dd0c9357 on vmware:master.


update to include required MABS server version

Please update this page to show that MABS V3 UR2 and later are required. per https://learn.microsoft.com/en-us/azure/backup/backup-mabs-protection-matrix#vm-backup

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 04e461b3-3829-51aa-dc02-2fc57d5d3662
Version Independent ID: 8c7e1a5a-ee87-f546-2ad3-56ae073de0eb
Content: Back up Azure Stack HCI virtual machines with MABS - Azure Backup
Content Source: articles/backup/back-up-azure-stack-hyperconverged-infrastructure-virtual-machines.md
Service: backup
GitHub Login: @jyothisuri
Microsoft Alias: jsuri

assigned to author

@DerekHerman-MSFT - thanks for writing to us, we are currently updating the articles with the support for the latest versions. These will be live soon.

@DerekHerman-MSFT - thanks for your suggestion. Added a link to the MABS support matrix in this section as per the Product Group's advise. The updates will be live soon.

#please-close


lift rules are confusing and/or broken

I am trying to run the full workflow. I end up with nearly half my reads in the unliftable BAM, and most of them do not fall within one of the un-liftable regions specified in the -R file.
When I set the -A and -q parameters to 0, then most of the un-liftable reads become liftable.
The documentation makes it sounds like -A and -q only determine whether a read goes to committed or deferred - they should not by themselves send any reads to the unliftable file. If this is not the case then this should be made more clear in the documentation, otherwise it seems there is a bug.

I opened this bug after looking at the intermediate outputs of the full workflow while it was running. Now that it's finished the uniftable file only has a small number of reads. Some of these are un-liftable due to being in un-liftable regions, and those appear in the final file as unmapped as I'd expect. However, there are some other reads in the un-liftable file that appear in the final file with alignment coordinates, indicating that they were re-mapped.
It would really help to have a more detailed explanation of how the full workflow works. It doesn't seem to match up exactly with what it says in the pre-print. Thanks

The logic used in the current workflow:

The *-unliftable.bam file: deferred + unliftable reads using the source coordinate system
The *-deferred.bam file: deferred reads using the target coordinate system

The logic is at: https://github.com/milkschen/leviosam2/blob/075575bc75309cd650926818125fe7e046c75426/src/leviosam.cpp#L261
I design this logic because the deferred reads in the source coordinate system could still be helpful. And it's costly to query such information from the original (unlifted) BAM.
The chain file can be converted into a BED file (my chaintools supports the operation), which can then be fed to bedtools intersect for the extraction of strictly unliftable alignments. I agree that this can be confusing. Would it help if I add a (optional) flag to only report strictly unliftable alignments in *-unliftable.bam?

Ok, this makes much more sense now. Maybe you could add these details to the README or a comment in the leviosam2.sh script?
Thanks!

That's a great suggestion. Thank you!
I updated the README (of the workflow) at https://github.com/milkschen/leviosam2/commit/412cb480c0e950371ec5a2842628c95f3b070cab


Implement caching for ruby dependencies used in Omnibus

What does this PR do?
Implement caching for ruby dependencies used on Omnibus (for Linux), reducing the reliance on rubygems or the caching mirror.
Motivation
Eventually drop the caching mirror.
Additional Notes
Windows and MacOS invoke omnibus somewhat differently and may be addressed in follow up work.
Example of cache hit: https://gitlab.ddbuild.io/DataDog/datadog-agent/-/jobs/498116809#L109
Possible Drawbacks / Trade-offs
Using $CI_JOB_NAME might be over-conservative (technically the platform would be the only thing that really matters), but it's used in this first implementation because it's simplest and safest.
Describe how to test/QA your changes

/merge


Fix autobuy

the whole time you have been autobuying improperly (near upgrade npc = attempt to purchase items). the issue was for i,v in next, (buyfunctions) do v(inv, currentupgrades, npctype and 'upgrade' or 'item') end
I replaced it with
for i,v in next, (buyfunctions) do v(inv, currentupgrades, npctype and 'item' or 'upgrade') end

I couldn't understand because oldvape code is really bad, I'll look into it before merging.

Cannot repro the upgrade npc issue, I have fixed the shop table.


Device Drop Down Menu

Hello Danny
Can You Give An Example , How To Make A Drop Down Menu That Shows Available Devices In Gui?
Any Help Is Appreciated
Natan

If Any One Reade This , Can You Guys Write this script For ME , Still No Response From Danny : (

Hi, It has already been implemented. In the icons tray, right click on the scripts icon => MIDI Input Devices => Here you can choose which one you want. BTW. in the original code, it is on line 234.

O Hey Matesaktesak And Thanks You For Your Help.
What I Try To Find Is To Make A Edit Box And Show The MIDI Input Devices In The Gui .


METRON-672: SolrIndexingIntegrationTest fails intermittently

This failure is due to a change in default behavior when indexing was split off into a separate configuration file.  The default batch size was changed from 5 to 1 in particular.  This, by itself, is not a problem, but the IndexingIntegrationTest (base class for Solr and Elastic search integration tests):

submits the configs
starts the indexing topology
writes the input data

The writing of the input data may happen before the topology fully loads or the configuration fully loads, especially if the machine running the unit tests is under load (like with travis).  As a result, the first record may end up with the default batch size (of 1) and write out immediately because the indexing configs haven't loaded into zookeeper just yet.  In that circumstance, eventually the configs load and the batch size is set to 5.  Meanwhile we've written 10 records and are expecting 10 in return, but because you wrote the first out already and then the next 5, we have another 4 pending to be written by the BulkMessageWriterBolt.
So, the failure scenario is as follows:

Message 1 is received and the indexing config hasn't loaded yet, so the batch size is 1 and it immediately gets written out
Message 2 - 5 are each received and the indexing config has loaded, so the batch size is 5 and it queues up
Message 6 is received and the batch writes out
Messages 7 - 10 are received, but never make a full batch, so we time out waiting for them to write out

The fix is to ensure that we don't write out messages to kafka until the configs are loaded, which is what this PR does.

To test this locally, because it's extremely sporadic that it happens locally (1 out of every 50 times I run the test), I did the following:

Built and installed the project in Maven: mvn -DskipTests clean install
Ran the metron-solr project integration tests for at least 2 hours in a row ensuring that they don't fail: echo "" > /tmp/output;while [ $(cat /tmp/output | grep "vs 6" | wc -l) -lt 1 ];do mvn install >& /tmp/output;done

Thanks for taking the effort to dig into this.  Great work.  Other than a couple minor comments, I'm very happy with this.

@justinleet comments addressed, let me know if there's anything else.


Climate: Sensibo fail to set temperature after 0.96 upgrade

Home Assistant release with the issue: 0.96
**Last working Home Assistant release (if known): 0.95.2
**Operating environment (Hass.io/Docker/Windows/etc.): Docker
**Component/platform: Sensibo climate component
https://www.home-assistant.io/components/sensibo/
Description of problem:
Following the upgrade to 0.96.0, it seems that the Sensibo climate platform is partially broken - I can't set the temperature, even from the developer tools. When trying to do so - an exception is thrown, see below:
Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):
- alias: LivingRoom AC - Temperature Up
  initial_state: on
  trigger:
    - platform: event
      event_type: xiaomi_aqara.click
      event_data:
        entity_id: binary_sensor.livingroom_ac_switch
        click_type: double
  action:
    - service: climate.set_temperature
      data_template:
        entity_id: climate.living_room_ac
        temperature: "{% if states.climate.living_room_ac %}{{ (state_attr('climate.living_room_ac', 'temperature')|round(0)) + 1 }}{%endif%}"

Traceback (if applicable):
2019-07-18 22:48:25 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection.139619050848464] None is not in list
Traceback (most recent call last):
  File "/usr/src/app/homeassistant/components/websocket_api/commands.py", line 121, in handle_call_service
    connection.context(msg))
  File "/usr/src/app/homeassistant/core.py", line 1150, in async_call
    self._execute_service(handler, service_call))
  File "/usr/src/app/homeassistant/core.py", line 1172, in _execute_service
    await handler.func(service_call)
  File "/usr/src/app/homeassistant/helpers/entity_component.py", line 194, in handle_service
    required_features
  File "/usr/src/app/homeassistant/helpers/service.py", line 316, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/app/homeassistant/helpers/service.py", line 339, in _handle_service_platform_call
    await func(entity, data)
  File "/usr/src/app/homeassistant/components/climate/__init__.py", line 497, in async_service_temperature_set
    await entity.async_set_temperature(**kwargs)
  File "/usr/src/app/homeassistant/components/sensibo/climate.py", line 277, in async_set_temperature
    index = self._temperatures_list.index(self.target_temperature)
ValueError: None is not in list

Additional information:

It looks like it is crashing because there is no current target temperature. Can you assume a state via the service before calling set temperature ?

@balloob I see you already made a fix for that, do you need anything on my end ?

See if things work really. I have not tested anything as I don't have access to Sensibo.

I can confirm that as of 0.96.3 everything’s back to normal, thanks, this
can be closed
On Mon, 22 Jul 2019 at 18:39 Paulus Schoutsen<EMAIL_ADDRESS>wrote:

See if things work really. I have not tested anything as I don't have
access to Sensibo.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/home-assistant/home-assistant/issues/25276?email_source=notifications&email_token=AHJP7PRLNP3ZI3TQTCU62ODQAXIETA5CNFSM4IE7ENL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2QKOSQ#issuecomment-513845066,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHJP7PQFJYQBTYLQWXDS5F3QAXIETANCNFSM4IE7ENLQ
.


Feature: add find duplication to sort-imports

What rule do you want to change?
sort-imports
Describe the problem
As far as I know, sort-import does not indicate when there are duplicate imports. For example:
Code example
import { X } from 'lib';
import { Y } from 'lib';
```Ï

### Additional comments

auto fix will be nice :)

### Validations

- [X] Read the [docs](https://perfectionist.dev).
- [X] Check that there isn't [already an issue](https://github.com/azat-io/eslint-plugin-perfectionist/issues) that reports the same bug to avoid creating a duplicate.

Duplicate #340


Updating to reflect proper navigation

Updating to reflect proper navigation

@JackStromberg : Thanks for your contribution! The author(s) have been notified to review your proposed change.

#sign-off

@JackStromberg: I'm sorry - only the author of this article, @Rajeswari-Mamilla, can sign off on your changes. But we do have an exception process - if you are on the Microsoft content or product team for this product area, you can ask the PR review team to review and merge it by sending mail to the techdocprs alias.

#sign-off


Log libStorage error from logVolumeLoopError

When libStorage would return an error like:
{
  "message": "POST https://api.digitalocean.com/v2/volumes: 422 Storage is not supported for this region",
  "status": 500,
  "error": "POST https://api.digitalocean.com/v2/volumes: 422 Storage is not supported for this region"
}

The only thing printed to the REX-Ray CLI output was
"error.status=500", and not the actual error message. Fix that by
asserting type to goof.HTTPError, then extracting the real message.
Fixes codedellemc/libstorage#429
This ends up being the difference between a final output of:
FATA[0002] error creating volume                         error.status=500 volume=test

vs
FATA[0002] error creating volume                         error.status=500 error.msg=POST https://api.digitalocean.com/v2/volumes: 422 Storage is not supported for this region volume=test

I think the new output is much more helpful

I admit I don't fully understand the interaction between golf, goof, and logrus. I tried for about 90 minutes to figure out why something like log.WithError(httpErr), whereby httpErr was created with httpErr, ok := err.(goof.HTTPErr) wouldn't automatically add error.msg to the log fields. error.status shows up on its own, but no other fields do.
I tried every combination of IncludeFieldsWith{Error,String,Format}, but still no luck. I could log the fields directly that way, by doing log.Debugf("%+v", httpErr), but log.WithError() would never show them.
I still think this is a vast improvement, and actually closes out the Epic #776.

Codecov Report

Merging #899 into master will decrease coverage by <.01%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##           master    #899      +/-   ##
=========================================
- Coverage     8.8%   8.79%   -0.01%
=========================================
  Files          25      25
  Lines        2977    2980       +3
=========================================
  Hits          262     262
- Misses       2679    2682       +3
  Partials       36      36

Impacted Files
Coverage Δ

cli/cli/cmds_10_volume.go
0.14% <0%> (-0.01%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 162c602...089ed93. Read the comment docs.


docs: Use GitHub instead of elixir

The broken link job fails often with elixir:
https://github.com/inspektor-gadget/inspektor-gadget/actions/runs/10391104393/job/28773219330

The job is failing because there is a docker broken link, I'll check it later on.


Code refactoring and tests issues

Branche: data_object
Last commit: bcfedd3cdb3b03f27e25ddca380b1b568fecdb48

Requested changes (after 13/01/2023 code review):
publish() methods must return an URL (string) :
I only changed the signature of all publish() methods. I am still looking for a way to implement them.
download() methods must return data:
They now return a list of bytes.
The list is not appropriate for blob_helper.create(). Because it only returns the bytes for one file.
On the other hand, it is for container_helper.create() and both are supposed to implement the "DataObject interface".
Does this approach seem appropriate to you?
Simplify the reading of unit tests:

I have added separators in comments for each sections
I created a functions module that provides the methods for both tests classes. It avoid code repetition.

I'm finally having trouble with other/new tests:
test CreateObject_PathNotExists_ObjectExists:
I defined two parameters for create() method: object_name and local_file_path.
In the context of a container, the second parameter is not required.
But this path help me with this test in the case of a blob: CreateObject_PathNotExists_ObjectExists
But I don't understand the meaning of this test in the context of a container.
TEST DeleteObject_ObjectContainingSubObjectsExists_ObjectDeletedRecursively:
I only know from the official documentation that the content of a container is deleted (all blobs).
But Azure Services Clients needs the objects (blob and client) to exist in order to be returned. The problem is: I need these client to check it.
Do I really have to implement it?

Changes:
Tests directory:

tests/functions
tests/blob/test_blob_helper.py
tests/container/test_container_helper

Src directory:

src/blob/blob_helper.py
src/container/container_helper.py
src/interface/data_object.py

I found a solution for this test (for blobs) DeleteObject_ObjectContainingSubObjectsExists_ObjectDeletedRecursively
If the user provides a path directory instead of a full blob name it will delete any blob that starts with this path.


(480) Add govuk-link class to roll out blog links

Add govuk-link class to roll out blog links

Before:

After:

 Bug: Fix banner link to use new design system css.

rebased.


Feature/355 category settings UI refactor part 2

Description:
This PR implements a new Edit Category / Edit Phrases UI flow - as specified by the latest Figma (see below for links).
These new screens were also included in the last iOS Vocable release - so feel to reference iOS during test as well.
New Screen 1: Edit Category Menu (after a user clicks Settings > Categories and Phrases > Clicks into a category to edit)
Figma: https://www.figma.com/file/mNrwUygVhTmuWnKNHHkenR/Vocable---Current?node-id=3356%3A17831

New Screen 2: Edit Category Phrases (after a user clicks "Edit Phrases" from the Edit Category Menu)
Figma: https://www.figma.com/file/mNrwUygVhTmuWnKNHHkenR/Vocable---Current?node-id=3356%3A17664

Acceptance Criteria:

[ ] User is able to edit & remove categories and have those changes reflected elsewhere in the app

[ ] User is able to hide & unhide categories and the category should appear at the end of the list when hidden

[x] User is able to edit & remove phrases and have those changes reflected elsewhere in the app

[ ] Screens in landscape mode are matching the design

[ ] Regression Testing

Issues - Remove Category:

After removing preset categories, when we are back to the main screen, the expected category and phrases are not present.
After removing a custom category, app crashed once.

Steps to Reproduce:

On the main screen, select a category
Go to the Settings, Categories and clicks Settings > Categories and Phrases > Click into this category to edit
Click on the Remove Category button
Come back to the main screen

https://user-images.githubusercontent.com/100222793/170111804-4e4243c3-7963-4027-906e-ddf4db689570.mov
Test Device:
Phone Emulator Android 12 (API 32)
Version:
V 1.5.1

Issue - Rename Category:

After renaming categories, when we are on the category details screen, the longer category name is not fitting to the screen.

Steps to Reproduce:

Go to the Settings, click Categories and Phrases
Select a category (Basic Needs)
Select Rename Category button to edit
Update the name (Basic Needs for evenings.)
Come back to the category details screen

https://user-images.githubusercontent.com/100222793/170118277-4bf9c1b1-8b7c-4042-9c5b-7c43b1c5d82a.mov

Design enhancement - Landscape mode:
After rotating the device to the landscape when we are on the category details screen;

Remove Category button is very close to the bottom.
Show Category button right margin can be adjusted according to the upper buttons.


Add documentation for iomete adapter

Description & motivation

Added iomete adapter. iomete is an all-in-one data infrastructure platform.
Related to dbt-iomete
Checklist
If you added new pages (delete if not applicable):

[x] The page has been added to website/sidebars.js
[x] The new page has a unique filename

2. All iomete specific settings already documented on iomete-profile.md

what about some features like indexes and "unlogged" tables? Here's today's versions of dbt-postgres configs and dbt-greenplum's configs (which is also somewhat postgres-based)

All iomete specific settings already documented on iomete-profile.md

what about some features like indexes and "unlogged" tables? Here's today's versions of dbt-postgres configs and dbt-greenplum's configs (which is also somewhat postgres-based)

Currently, no additional configuration is required to work with dbt-iomete. We've fixed the first Iceberg-specific issue and added it to the documentation. We will update the documentation if different situations arise after we start working with dbt.


GitHub stream

Do i have to add migration on my branch as well to pass the tests? @kylecarlstrom

Not sure what about migrations issues. @TianZhiWang @nettapper can you two take a look?


[Bug]: CA handling is not working

Please add the exact image (with tag) that you are using
eclipse-temurin:11-jdk
Please add the version of Docker you are running
Kubernetes
What happened?
Created a ConfigMap that includes CA file (MyOrg-RootCA.crt in my example), then mounted it to /certificates directory as described on Docker Hub. Then I start the pod using Kustomization below.
I found __cacert_entrypoint.sh when I entered the container with shell, then I see below output when I run it.

Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.

After some investigation I found following contents in /usr/local/share/ca-certificates/.

drwxr-xr-x 1 root root 4096 Feb  8 03:49 .
drwxr-xr-x 1 root root 4096 Feb  2 07:42 ..
lrwxrwxrwx 1 root root   23 Feb  8 03:19 MyOrg-RootCA.crt -> ..data/MyOrg-RootCA.crt

It's broken symlink. Maybe need to use cp -s instead of cp -a.
Here is the contents of /certificates

drwxr-xr-x 2 root root 4096 Feb  8 03:19 ..2024_02_08_03_19_14.996191449
lrwxrwxrwx 1 root root   31 Feb  8 03:19 ..data -> ..2024_02_08_03_19_14.996191449
lrwxrwxrwx 1 root root   23 Feb  8 03:19 MyOrg-RootCA.crt -> ..data/MyOrg-RootCA.crt

kustomization.yaml
configMapGenerator:
  - name: cacerts
    files:
      - MyOrg-RootCA.crt

resources:
  - pod.yaml

pod.yaml
apiVersion: v1
kind: Pod
metadata:
  name: openjdk
spec:
  containers:
  - name: openjdk
    image: eclipse-temurin:11-jdk
    command: [ "/bin/bash", "-c", "--" ]
    args: [ "while true; do sleep 30; done;" ]
    volumeMounts:
      - name: cacerts
        mountPath: /certificates
    env:
      - name: USE_SYSTEM_CA_CERTS
        value: "1"
  volumes:
    - name: cacerts
      configMap:
        name: cacerts

Relevant log output
No response

@rassie hello, could you please see this issue. Thank you

Is the problem that ..data/MyOrg-RootCA.crt should be ..**/**data/MyOrg-RootCA.crt?

it's copying symlink as is, but pointing file doesn't exists. it works find with cp -s

I could open a PR with a with, but would like to discuss proper solution for it.

Oh, that's not nice, I should have seen that coming. In principle, cp -a should be fine, we just don't consider dotted files when copying, but I think ..data is a mount-point, which would not be copied correctly. The solution might be as simple as making a cp -aL out of cp -a.
I'm a bit time-constrained at the moment, someone care to make a PR with cp changed and a bit of symlink magic in .test/tests/java-ca-certificates-update/?

@dulmandakh - Thanks, we'll get round to this, but PR's are welcome as well :-)

@dulmandakh - Thanks, we'll get round to this, but PR's are welcome as well :-)

Hello, in Mongolia we're celebrating lunar new year, so I won't be able to work for next 3 4 days. Please feel free to create a fix.

My workaround for now:
    volumeMounts:
      - name: cacerts
        mountPath: /certificates/MyOrg-RootCA.crt
        subPath: MyOrg-RootCA.crt


Pagination

thanks @farbodahm
for now, it is good but for the future, we should consider that reqparser is going to be deprecated:
https://flask-restful.readthedocs.io/en/latest/reqparse.html


plot_tree for multiclass crashed

I'm using the new v2.0 xgboost for multiclass regression and the function plot_tree crashes.
This is an example inspired in the documentation https://xgboost.readthedocs.io/en/stable/python/examples/multioutput_regression.html
Example
import numpy as np
import xgboost as xgb

rng = np.random.RandomState(1994)
X = np.sort(200 * rng.rand(100, 1) - 100, axis=0)
y = np.array([np.pi * np.sin(X).ravel(), np.pi * np.cos(X).ravel()]).T
y[::5, :] += 0.5 - rng.rand(20, 2)
y = y - y.min()
y = y / y.max()

reg = xgb.XGBRegressor(
    tree_method="hist",
    n_estimators=128,
    n_jobs=16,
    max_depth=8,
    multi_strategy="multi_output_tree",
    subsample=0.6,
)
reg.fit(X, y, eval_set=[(X, y)])

xgb.plot_tree(reg)

It's a highly experimental feature as noted in the demo. Anything not shown in that demo can be assumed to be not yet supported.

Closing in favor of https://github.com/dmlc/xgboost/issues/9043 .

Just to clarify, xgboost works with classification since day 1. The issue here is using regression with multi_output_tree.

Ok, thanks! I will wait until it is implemented!


github docs pages

Document the SQL API. Basically everything that's already in the readme, but provide full detail into the function parameters, return types, etc.
It would be nice if the client SDKs could also live in the github pages site. However, the Rust SDK already has decent hosting at docs.rs. The python SDK does not have any docs yet.
Some topics that should be covered to start:

installation. e.g. from source and trunk install. Currently in the contributing guide.

main public SQL API. vast majority of this work already lives in the main readme

the rust specific APIs (PGMQueue and PGMQueueExt) can continue to live in docs.rs, but ideally live in pgmq docs.

PGMQueueExt API currently has basically no doc

metrics - what metrics exist, and how to view them.

patterns and implementations - quick-start for 1-2 basic applications that use pgmq

Let's break this task up into a few different sub-issues.

closed https://github.com/tembo-io/pgmq/pull/150


Test pilot page is deprecated

https://doc.owncloud.org/server/9.2/developer_manual/testing/index.html
It has former emails and perhaps an old description.

Something for carlaschroder ?

@carlaschroder

@guruz Thanks, I'll remove it. Which ownCloud releases does this apply to, 9.2 only?

@settermjd  :)

PR here #2887


⚠ Bofrost.be has degraded performance

In a579bb0, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 901 ms

Resolved: Bofrost.be performance has improved in 7bc0c94.


IndexMut for Map and Value

Fixes #247.

let mut data = json!({ "x": 0 });

// replace an existing key
data["x"] = json!(1);

// insert a new key
data["y"] = json!([false, false, false]);

// replace an array value
data["y"][0] = json!(true);

// inserted a deeply nested key
data["a"]["b"]["c"]["d"] = json!(true);

David this is awesome. Also I think you could be telepathic. You have managed to help without me even asking you. Thanks.
All misspelling thanks to my iPhone.
On Thu, Feb 9, 2017 at 9:31 AM -0500, "Oliver Schneider"<EMAIL_ADDRESS>wrote:
Merged #249.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.


Problem - data format

We will need to download the files from linked in and ensure they are in formats that can be manipulated by python. If they can't we'll need to reformat the data. Need to be clear on what exact formats are compatible.

Went to LinkedIn and successfully downloaded data files in csv format. This format is acceptable for Python so problem has been resolved and can be closed.


fix: inherit tabstop on context window

Addresses #146
Sets tabstop same as the current buffer.

Tested. Works fine in local machine


URL link broken

When click on  "TinyFPGA programmer" in prerequisites section in README.md file, it doesn't take me to the correct URL.

closing this issues, as it is resolved as part of https://github.com/QuickLogic-Corp/qorc-sdk/pull/3