Make Broadcom WLAN work (e.g., for MacBooks)

https://unix.stackexchange.com/questions/595446/configuring-bcm4360-macbook-air-on-freebsd-to-connect-to-wifi

https://github.com/helloSystem/ISO/commit/93218c55ff02a92818ac420a762a6142a2a82a83

https://github.com/helloSystem/ISO/commit/93218c55ff02a92818ac420a762a6142a2a82a83

Argh, https://forums.freebsd.org/threads/installing-broadcom-bcm43236-wifi-on-11-3-missing-firmware-error.76470/ says

There is no package for net/bwm-firmware-kmod, you need to build this from ports.

Maybe we can get the firmware from the existing Ubuntu package.

Argh, https://forums.freebsd.org/threads/installing-broadcom-bcm43236-wifi-on-11-3-missing-firmware-error.76470/ says

There is no package for net/bwm-firmware-kmod, you need to build this from ports.

Maybe we can get the firmware from the existing Ubuntu package.

https://steemit.com/freebsd/@valkyr/freebsd-on-lenovo-ideapad-s10-feat-broadcom-bcm4312
Seems to be quite involved to get this thing running!

https://steemit.com/freebsd/@valkyr/freebsd-on-lenovo-ideapad-s10-feat-broadcom-bcm4312
Seems to be quite involved to get this thing running!

Also see https://github.com/helloSystem/ISO/commit/93218c55ff02a92818ac420a762a6142a2a82a83

Also see https://github.com/helloSystem/ISO/commit/93218c55ff02a92818ac420a762a6142a2a82a83

The 4312 does work on NomadBSD with some lines added to the loader.conf. I replaced the card with an Atheros, but I have a laptop where it is easy to switch, so I could do some testing once the firmware is added.

once the firmware is added

How would one do that in the most straightforward way? (How does it work on NomadBSD?)

pkg info on my nomad system shows: bwn-firmware-kmod-0.1.2 (bwi is for an older broadcom chipset).
You may need to install from ports due to the license, not sure I haven't tried on pure FreeBSD.
The nomad system has bwn_v4_ucode.ko, bwn_v4_lp_ucode.ko, bwn_v4_n_ucode.ko in /boot/modules.
I added the following to /boot/loader.conf:
if_bwn_load="YES"
bwn_v4_ucode_load="YES"
bwn_v4_lp_ucode_load="YES"
I was going to try just copying the modules over and editing the loader.conf, but it appears modifying the run from usb system is non trivial. I might be able to scrounge anpther laptop HDD to try a full install on.

Hint: You can also install helloSystem to a >= 8 GB USB stick rather than to an internal hard drive (I am typing this from one).

I got a laptop I could do an install to the HDD on and tested with the Broadcom 4312 card. This card is a b/g, but I also have a 4313, which is the n version. I tried fetching ports, but the bwn-firmware-kmod port requires kernel source, so I stopped there. It looks like it also requires the b43-fwcutter port. I installed from 0.5.0_OE141-FreeBSD-12.2, so I copied the .ko files from my Nomad system (also 12.2 based) into /boot/modules. I added if_bwn_load="YES" and bwn_v4_lp_ucode_load="YES" to the loader.conf and rebooted and was able to configure the card and connect to the network. I tried  bwn_v4_ucode_load first, but the error messages showed it was looking for the lp firmware. I'm on that laptop now. pciconf shows:
bwn_pci0@pci0:12:0:0:   class=0x028000 card=0x000c1028 chip=0x431514e4 rev=0x01 hdr=0x00
vendor     = 'Broadcom Inc. and subsidiaries'
device     = 'BCM4312 802.11b/g LP-PHY'
class      = network
snippet from ifconfig:
parent interface: bwn0
media: IEEE 802.11 Wireless Ethernet OFDM/48Mbps mode 11g
status: associated

Thanks for testing @dotdash70. Can you please give the exact names of the files you copied?

Additional Notes:
I had to manually add the wlan entries to /etc/rc.conf to get the adapter to come up. I also copied over the working wpa_supplicant.conf from the Nomad system.
I tried the 4313 card, and that one is not supported under FreeBSD.
Found some more detailed information on Broadcom wireless cards in FreeBSD here:
https://landonf.org/code/freebsd/Broadcom_WiFi_Improvements.20180122.html
Most of that was over my head, but it appears newer chipsets require the FullMAC driver, which is not currently implemented.
This may happen in FreeBSD 13.x as work on SDIO progresses. Source: https://wiki.freebsd.org/SDIO

If one has Ports installed:
sudo su
cd /usr/ports/net/bwn-firmware-kmod && make install

Source: https://unix.stackexchange.com/a/595474

On Mac mini Mid 2010 (Macmini4,1) helloSystem 0.7.0 getting KLD bwn:v4_ucode.ko_ depends on kernel - not available or version mismatch.
TODO: Check whether /boot/modules/bwn_v4_ucode.ko exists.

On Mac mini Mid 2010 (Macmini4,1) helloSystem 0.7.0 getting KLD bwn:v4_ucode.ko_ depends on kernel - not available or version mismatch.
TODO: Check whether /boot/modules/bwn_v4_ucode.ko exists.

https://wiki.freebsd.org/IntelMacMini suggests (for MacMini3,1):
  if_bwn_load="YES"
  bwn_v4_ucode_load="YES"
  bwn_v4_n_ucode_load="YES"

When doing this on FreeBSD 13.0-RELEASE based helloSystem 0.7.0, the boot loader complains about kernel being the wrong version.

https://wiki.freebsd.org/IntelMacMini suggests (for MacMini3,1):
  if_bwn_load="YES"
  bwn_v4_ucode_load="YES"
  bwn_v4_n_ucode_load="YES"

When doing this on FreeBSD 13.0-RELEASE based helloSystem 0.7.0, the boot loader complains about kernel being the wrong version.

Coming back to do some testing with 7 G160 version based on FreeBSD 13.0
My older laptop has had issues installing from the 5 version on, though it's booting the live system nicely now. I tried this on a SFF desktop with a mini-pcie slot that I can put the old Broadcom card in. I decided to try it the hard way this time, rather than copy the files from my NomadBSD system. First I tried from my fresh Hello System 7 install, which was running fine with an Intel wireless card. First I installed ports, then found I needed source, so downloaded and extracted the src tree. When trying to make the b43-fwcutter port, I found the system wasn't finding cc. I tried symlinking clang, but the make was still blowing up, so I did a quick install of vanilla FreeBSD 13.0 on another box, installing ports and src. From that box, I was able to make b43-fwcutter,  and bwn-firmware-kmod from the ports tree. I copied bwn_v4_ucode.ko, bwn_v4_n_ucode.ko, and bwn_v4_lp_ucode.ko from the FreeBSD system to /boot/modules/ on my HelloSystem. The added lines to boot/loader.conf:
if_bwn_load="YES"
bwn_v4_ucode_load="YES"
bwn_v4_n_ucode_load="YES"
bwn_v4_lp_ucode_load="YES"
and etc/rc.conf:
wlans_bwn0="wlan1"
create_args_wlan1="down country US"
ifconfig_wlan1="up scan WPA DHCP"
I was then able to shut the HelloSystem machine down, swap the wireless card, and have it come up working on the broadcom card:
ifconfig wlan1
wlan1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
ether 00:26:5e:00:aa:bb
inet6 fe80::226:5eff:fe00:aabb%wlan1 prefixlen 64 scopeid 0x3
inet <IP_ADDRESS> netmask 0xffffff00 broadcast <IP_ADDRESS>
groups: wlan
ssid TEST channel 11 (2462 MHz 11g) bssid 24:a4:3c:00:aa:bb
regdomain FCC country US authmode WPA2/802.11i privacy ON
deftxkey UNDEF AES-CCM 3:128-bit txpower 30 bmiss 7 scanvalid 60
protmode CTS wme roaming MANUAL
parent interface: bwn0
media: IEEE 802.11 Wireless Ethernet OFDM/24Mbps mode 11g
status: associated
nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
I can supply the FreeBSD 13.0 versions of the .ko files (and bwi_v3_ucode.ko that I built since I was at it) if needed.

Thanks @dotdash70 for taking the time to check this out.

When trying to make the b43-fwcutter port, I found the system wasn't finding cc.

By default, helloSystem does not ship compilers and other developer-related tools. To get these, you need to install the Developer Tools from developer.img on https://github.com/helloSystem/ISO/releases/tag/r0.7.0 as well.

I can supply the FreeBSD 13.0 versions of the .ko files (and bwi_v3_ucode.ko that I built since I was at it) if needed.

This would be very helpful. You can put them into a zip and attach them here. Thanks!
Could you also create pkg packages for b43-fwcutter and bwn-firmware-kmod using sudo make package && ls work/pkg?

Here are the .ko files built from a FreeBSD 13.0-Release system:
bw-ko-fbsd13.zip
As for creating a pkg, do you mean running that command on FreeBSD 13, or on helloSystem? If helloSystem, then I'd imagine the procedure would go something like- download the developer.img, install Developer Tools, fetch kernel sources, fetch ports, make/install the bw43-fwcutter port, make/install bwn-firmware-kmod, then use the make package command for both ports. Does that sound correct? I've never tried to pkg a port. Additionally, I think the reason there is not a pkg for the firmware files is due to the licensing. The build process grabs the files from openwrt, iirc.  Recent versions of GhostBSD and NomadBSD are based on FreeBSD 13.0 and have the .ko modules present in boot/modules, so their developers may have some insight on the particulars of bundling the files with a distribution.

Thanks @dotdash70. This is very helpful.
With making the pkg, i meant on FreeBSD 13. (helloSystem 0.7.0 is a preconfigured FreeBSD 13 with some additions, after all.)

Had to edit the makefile as detailed earlier. Here is the result
pkg.zip

Thank you very much @dotdash70.

I've tried loading the Broadcom driver for the BCM4321 on my MacBook 4,1, but it doesn't work. /var/log/messages says bwn_phy_n_attach: BWN_GPL_PHY not in kernel config; no PHY-N support and then says that it failed. Any idea how to fix this? The most plausible one seems to be recompiling the kernel, but I have no idea how to do that on FreeBSD (and the documentation, as far as I've seen, is quite terrible compared to Linux).

@Slater91 did you install the package from @dotdash70?

Yes, I have. I have read multiple discussions where people specifically said that you have to enable the flag in the kernel in order for the driver to work. I'll post the links as soon as I find them again.
My loader.conf has this:
if_bwn_load="YES"
bwn_pci="YES"
bwn_v4_ucode_load="YES"
bwn_v4_n_ucode_load="YES"
bwn_v4_lp_ucode_load="YES"

Here is the /var/log/messages log:
messages.txt

I have read multiple discussions where people specifically said that you have to enable the flag in the kernel in order for the driver to work.

Did you find any reason why this flag is apparently not enabled by default in the FreeBSD kernel then?
In helloSystem we want to use the official kernels provided by the FreeBSD project. So my question is, why is the FreeBSD project not setting the flag in the kernel.

Unfortunately, my test kit consists of a 4312 (works) and a 4313 (does not work). A quick search seems to indicate the 4321 may not be supported, or may require additional steps. Landon Fuller's page shows only the dual-band version being supported under bwn, with the caveat 'The optional bwn(4) PHY driver is derived from b43 GPL code, and must be explicitly enabled.'  I'm not sure exactly what that entails, having never had to do it. I'm not sure what deficiencies you are finding in the FreeBSD docs, but I have always found them very helpful. Kernel configuration is right in the handbook. https://docs.freebsd.org/en/books/handbook/kernelconfig/ If helloSystem is using the GENERIC kernel, you could use that as a base, but if there are any customizations in the kernel, you would need to include those as well.

Came across this post with detailed instructions on how to compile a kernel with BWN_GPL_PHY here:
https://forums.freebsd.org/threads/bcm4322-wifi-card-freebsd-11-2-not-working.68103/
I would expect there are licensing issues keeping it from being in the GENRIC kernel.

I would expect there are licensing issues keeping it from being in the GENRIC kernel.

That is what I was fearing.
What I don't understand, though:
Why does the Broadcom kernel module need a special configuration (apparently with Broadcom proprietary parts?) of the kernel itself in order for it to work? Isn't the whole point of a kernel module to keep certain things out of the kernel itself?

A bit beyond me, but from looking over Landon's page, the Broadcom chipsets seem to have a lot of variation of parts, and some currently only work with by replacing the PHY bits with those borrowed from the Linux GPL version. I guess that needs to be done at the kernel level. https://svnweb.freebsd.org/base?view=revision&revision=326841

I can confirm that enabling "BWN_GPL_PHY" in the kernel build options leads to a working WiFi chip.

I'm not sure what deficiencies you are finding in the FreeBSD docs, but I have always found them very helpful. Kernel configuration is right in the handbook. https://docs.freebsd.org/en/books/handbook/kernelconfig/ If helloSystem is using the GENERIC kernel, you could use that as a base, but if there are any customizations in the kernel, you would need to include those as well.
helloSystem comes without kernel sources and finding instructions on how to download them wasn't easy, also due to the (relatively recent) change from SVN to Git. Most discussions I've found said "if you don't have the sources installed there's something wrong and you probably need to reinstall", which is not really helpful. In general, what I've seen of FreeBSD documentation is either very basic, or incomplete, or assuming that you already know most things, which is incredibly hostile towards newcomers. I don't want to start an in-depth discussion here about this, I'm just reporting my experience as a Linux system administrator with some experience.

https://twitter.com/Ericbsd/status/1488879439731695623

With the new update I made yesterday, I got my iMac BCM4322 wifi working with net/bwn-firmware-kmod and bwn_v4_n_ucode_load="YES" in loader.conf.

That’s awesome! And no GPL code gets included in the kernel?

Oh yeah the GPL code is included.

Does this mean that technically it works but there is no way to distribute the working kernel in binary form legally?

A major issue I've found is that the driver randomly stops working. It simply brings everything down and I have found no way to make it work without a reboot; it would just try to connect and fail ("4-way handshake failed, the secret provided is not correct" without any changes made to anything!), then rinse and repeat ad libitum. I've found similar issues on the FreeBSD forums with no solutions.
Adding to this, the WiFi connection only works with 2.4 GHz networks on my MacBook; if I try to connect to 5 GHz networks, the machine simply reboots leaving nothing in the logs to understand why. I won't even touch on theme of Bluetooth.
Frankly speaking, seeing these issues on such basic things as WiFi and Bluetooth, on a very popular 14-year-old machine no less!, is making me question the choice of FreeBSD as the base system, mainly because I don't see the very significant improvements needed to these areas coming any time soon. If this is to be an operating system that "just works" and that appeals to the masses, the amount of work involved just to take FreeBSD to the twenty-first century in terms of hardware and features support is going to be mastodontic and I am not sure it can realistically be achieved. I don't want to be pessimistic, but my experience so far has been less than stellar on multiple machines (with FreeBSD, not with the strictly helloSystem parts such as helloDesktop) and it seems there are no solutions in sight.

https://twitter.com/Ericbsd/status/1488879439731695623

With the new update I made yesterday, I got my iMac BCM4322 wifi working with net/bwn-firmware-kmod and bwn_v4_n_ucode_load="YES" in loader.conf.

That’s awesome! And no GPL code gets included in the kernel?

Oh yeah, the GPL code is included.

Does this mean that technically it works, but there is no way to distribute the working kernel in binary form legally?

@probonopd I don't think it is illegal, but to have most BWN drivers working correctly, you have to build the kernel with BWN_GPL_PHY in the GENERIC, and vanilla FreeBSD tries to avoid having GPL code installed by default.
It is one of the reasons I compile my own FreeBSD source code for GhostBSD.

As far as I know, distributing it is at the very least legally dubious. You would effectively be distributing GPL code as a fundamental part of a program that was licensed as BSD, which theoretically means that the whole code base would be relicensed to GPL. Which is, from a legal perspective, a bit of a nightmare. This is why vanilla FreeBSD tries to avoid having GPL code installed by default, because the GPL is more restrictive and it's viral.

If we are using the driver provided by open BSD, this is seen from the Internet.
https://streof.github.io/freebsd-wifi-mac/
https://man.openbsd.org/bwfm

Status of this driver:

https://forums.freebsd.org/threads/broadcom-fullmac-wifi-integration.84991/
https://landonf.org/code/freebsd/Broadcom_WiFi_Improvements.20180122.html

https://forum.nomadbsd.org/t/1-3-1-mac-on-macbook-pro-2011-17/32
well, maybe this one works too...

So, does it work on NomadBSD now?


Logging is not published to Application Insights if configure_azure_monitor is called in another module.

Package Name: azure-monitor-opentelemetry
Package Version:  1.1.1
Operating System: MacOS, Debian (bullseye)
Python Version: 3.11

Questions
I'm new to OpenTelemetry and Django so my usage might be wrong but when I'm instrumenting Django app, logging does not get published to Application Insights if python's logging API is called in the different module file from one that calls configure_azure_monitor().
In my app, firstly it's called in Django's setting module settings.py.
if os.getenv('APPLICATIONINSIGHTS_CONNECTION_STRING'):
    print('Enabling OpenTelemetry...')
    from azure.monitor.opentelemetry import configure_azure_monitor

    configure_azure_monitor()
    logger = logging.getLogger(__name__)
    logger.warning('OpenTelemetry has been enabled')
else:
    print('Environment variable APPLICATIONINSIGHTS_CONNECTION_STRING is not set. Skipping OpenTelemetry configuration')

The warning log OpenTelemetry has been enabled was sent to traces table in Application Insights. But it wasn't published in another module. The example below emits exception logs but it's not shown up in Application Insights, regardless the configure_azure_monitor() is called or not in the module.
from rest_framework.views import APIView
from rest_framework.response import Response
from azure.monitor.opentelemetry import configure_azure_monitor

import os
import logging

# configure_azure_monitor()
logger = logging.getLogger(__name__)

class ExceptionTraceViews(APIView):
    permission_classes = []
    authentication_classes = []

    def post(self, request):
        data = request.data
        try:
            int('aaa')  # Exception as it tries to convert non-numeric string to integer
        except:
            message = f"{data.get('exception').get('message')}"
            logger.exception(data.get('exception').get('message'), extra={'custom_dimensions': {'api': 'logging', 'type': 'exception_telemetry'}})
        return Response({'message': 'OK'}, status=200)

And if configure_azure_monitor() is called, the following warning messages are shown and sent to Application Insights so I assumed calling configure_azure_monitor() multiple times is not ideal.
WARNING Overriding of current TracerProvider is not allowed
WARNING Overriding of current LoggerProvider is not allowed
WARNING Overriding of current MeterProvider is not allowed
WARNING Attempting to instrument while already instrumented
WARNING Attempting to instrument while already instrumented
WARNING Attempting to instrument while already instrumented
WARNING Attempting to instrument while already instrumented
WARNING Attempting to instrument while already instrumented

As far as I read the source code, it looks that OTEL configuration is set to root logger so I thought it's OK to use it without calling configure_azure_monitor() multiple times. What is correct practice to configure instrumentation with azure-monitor-opentelemetry if it is used in multiple modules.
To Reproduce
Steps to reproduce the behavior:

call configure_azure_monitor() in one module and call logging API in another module.

Expected behavior
Ideally, once configure_azure_monitor() is called, logging is published. Or if some specialized logger is configured within the library then I wish to know how to retrieve it.

Thanks for the feedback, we’ll investigate asap.

@shingotoda
Thanks for using the Azure monitor OpenTelemetry Distro. You are correct in assuming that configure_azure_monitor() should not be called multiple times. As for the missing logging from ExceptionTraceViews, I was not able to replicate the issue. I modified the basic django app to include configure_azure_monitor() in the settings.py file and was able to capture logging telemetry with logger.warning sent from settings.py as well as views.py (I removed configure_azure_monitor() from views.py). As for best practices, I would recommend the following:

Place the configure_azure_monitor() call in the views file in which you will be making the majority of your request calls that you want to track.
Only call configure_azure_monitor() once.
To only capture logs sent from logging.X(...) calls that only belong to your app, set the root logger that you want to capture in the configuration for configure_azure_monitor() as shown below:

...
configure_azure_monitor(logger_name="<name-of-logger-you-want-to-track>")
...

Note that any logger that is a children of the logger you put in logger name will also be tracked (just like how Python standard logging works).

@lzchen Thank you for the suggestion. I think I managed to make it work to capture logs. I found that there is a root logger setting in my settings.py as below.
    "root": {
        "handlers": ["console"]
    }

If I remove this, then it starts capturing logs in another module. Do you think the root logger setting in settings.py harmful?
Also I found that logger.exception is not captured though I thought this is at ERROR level. Is this a limitation?

This would not be helpful for local development.

It is fine to have it setup the way you are currently if you don't mind the excessive noise, because you will be capturing logs from the distro itself if you do add it to the root logger.

Also I found that logger.exception is not captured though I thought this is at ERROR level. Is this a limitation?

Did you check the exceptions table? logger.exception() calls wind up in that table instead of the traditional traces table.

@lzchen

Did you check the exceptions table? logger.exception() calls wind up in that table instead of the traditional traces table.

exception itself is shown in the exceptions table but message sent by logger.exception() is not shown up in neither exceptions nor traces tables. But it's a minor issue so I'm OK with this.

It is fine to have it setup the way you are currently if you don't mind the excessive noise, because you will be capturing logs from the distro itself if you do add it to the root logger.

Understood that recommendation is to use named logger rather than root. But I saw some behaviors that I do not really understand with named logger approach.
Now I call configure_azure_monitor with arg logger_name='azure' like this.
    from azure.monitor.opentelemetry import configure_azure_monitor
    configure_azure_monitor(logger_name='azure')

And configure root logger like this so that it will emit logs >=INFO to console.
    "root": {
        "handlers": ["console"],
        "level": 'INFO'
    }

And instrumentation is like this. See that now it get logger with name azure.
logger = logging.getLogger('azure')

class ExceptionTraceViews(APIView):
    permission_classes = []
    authentication_classes = []

    def post(self, request):
        data = request.data
        try:
            int('aaa')
        except:
            logger.exception('exception: ' + data.get('exception').get('message'), extra={'value': '123'})  # exception is NOT sent
            logger.critical('critical: ' + data.get('exception').get('message'))  # critical is sent
            logger.error('error: ' + data.get('exception').get('message'))        # error is sent
            logger.warn('warn: ' + data.get('exception').get('message'))        # warn is sent
            logger.info('info: ' + data.get('exception').get('message'))            # info starts to be sent when root's log level is INFO
            logger.debug('debug: ' + data.get('exception').get('message'))   # debug is NOT sent
        return Response({'message': 'OK'}, status=200)

Firstly, it starts emitting a lot of the following logs and they are ingested to Application Insights traces table. How can I prevent this?
INFO Request URL: 'https://japaneast-1.in.applicationinsights.azure.com//v2.1/track'
Request method: 'POST'
Request headers:
    'Content-Type': 'application/json'
    'Content-Length': '3163'
    'Accept': 'application/json'
    'x-ms-client-request-id': '40e674f0-9b1f-11ee-b6b1-da23c070c806'
    'User-Agent': 'azsdk-python-azuremonitorclient/unknown Python/3.11.2 (macOS-14.2-arm64-arm-64bit)'
A body is sent with the request
INFO Response status: 200
Response headers:
    'Content-Type': 'application/json; charset=utf-8'
    'Server': 'Microsoft-HTTPAPI/2.0'
    'Strict-Transport-Security': 'REDACTED'
    'X-Content-Type-Options': 'REDACTED'
    'Date': 'Fri, 15 Dec 2023 07:55:02 GMT'
    'Content-Length': '49'
INFO Transmission succeeded: Item received: 3. Items accepted: 3

Secondary, the name logger starts to send INFO logs even though I did not set log level to this named logger. What I did is to set INFO log level to the root logger. I thought default log level is WARN for the named logger so INFO logs wouldn't be sent. How can I control log level to be sent to Application Insights?

Now I call configure_azure_monitor with arg logger_name='azure' like this.

Please name your logger something that doesn't start with azure. Our SDKs are under the azure namespace which is why you are most likely capturing logs sent by the SDK as well :).

Secondary, the name logger starts to send INFO logs even though I did not set log level to this named logger.

The named logger takes the configuration from the ROOT logger unless overridden. Set the log level to WARN explicitly for your named logger to get WARN level messages and above.

@lzchen Thank you for the suggestion. OK, I will take another logger name not starting from azure. Also I understand that log level should be explicitly set to a named logger. But I still see the following message is emitted to stdout then results in being ingested to Log Analytics workspace.
INFO Request URL: 'https://japaneast-1.in.applicationinsights.azure.com//v2.1/track'
Request method: 'POST'
Request headers:
    'Content-Type': 'application/json'
    'Content-Length': '893'
    'Accept': 'application/json'
    'x-ms-client-request-id': 'e164e506-9b7e-11ee-94dd-92dfbd47e697'
    'User-Agent': 'azsdk-python-azuremonitorclient/unknown Python/3.11.7 (Linux-5.15.0-1052-azure-x86_64-with-glibc2.31)'
A body is sent with the request
INFO Response status: 200
Response headers:
    'Content-Type': 'application/json; charset=utf-8'
    'Server': 'Microsoft-HTTPAPI/2.0'
    'Strict-Transport-Security': 'REDACTED'
    'X-Content-Type-Options': 'REDACTED'
    'Date': 'Fri, 15 Dec 2023 19:19:33 GMT'
    'Content-Length': '49'
INFO Transmission succeeded: Item received: 1. Items accepted: 1

After I debugged with pdb, it looks that this line is processed when telemetry is sent to Application Insights.
https://github.com/Azure/azure-sdk-for-python/blob/d8bcdc57c0287dca6b6bdc192c9255616c4f36d3/sdk/core/azure-core/azure/core/pipeline/policies/_universal.py#L509C1-L512C23
            if http_request.body:
                log_string += "\nA body is sent with the request"
                logger.info(log_string)
                return

This logger has root as parent.
(Pdb) logger.name
'azure.core.pipeline.policies.http_logging_policy'
(Pdb) logger.parent
<RootLogger root (INFO)>

I would think that since this message is too verbose from application perspective and it takes root logger, this should call logger.debug() like other lines of this code like this.
https://github.com/Azure/azure-sdk-for-python/blob/d8bcdc57c0287dca6b6bdc192c9255616c4f36d3/sdk/core/azure-core/azure/core/pipeline/policies/_universal.py#L324
Though it, I'm new to this codebase so my assumption might be wrong. Please let me know if there is a workaround to suppress this message.

@lzchen I think this issue is expanding from original question so I raised separate issue to discuss the verbose log.
https://github.com/Azure/azure-sdk-for-python/issues/33623
Please feel free to close this issue.


Feature/ubuntu 20.4

@mithundiddi @jpsnir Please review and merge. OpenCV update for ubuntu 20.04

Why is the CI failing, it should continue working with the old versions right?

Not sure, Re-running the build.

Found the issue, it is failing because travis build is happening for ubuntu < 20.04 with opencv 3.2.0 and this branch is particularly for opencv 4.x and ubuntu 20.04. We have to setup build for this configuration.

One thought, all these ifdefs make it really hard to read the code.  How about we reaname all the calls to the new open cv functions.  Then in one place do a OpenCV version check and then assign function aliases to the old opencv functions if needed. This way there should only be 1 if def.  Aliases: https://www.fluentcpp.com/2017/10/27/function-aliases-cpp/

@PushyamiKaveti @mithundiddi this should fix the issue building.  Please try it out once.


Development

#issue #3051 fixed.

Hi, please, I am having this problem after following your aws install guide. Please how to I merge the fix into my code ? Or is there a tutorial of which I can get a clean install without having to merge ? I used create project bagisto/bagisto. Thank you in advance for your kind attention and thanks for bagisto. thanks.


Add standard font support back to pdfkit

Custom font substitution engine built to support this: https://github.com/react-pdf/font-substitution


'SKPhoto' initializer is inaccessible due to 'internal' protection level

'SKPhoto' initializer is inaccessible due to 'internal' protection level

Same issue here.
Add public to SKPhoto.init() ?


App wallpaper behind NavBar

I'm trying to achieve something similar to #64 and the Thousand Ways to Navigate in React Native example but don't appear to be able to.
I've created a simplified Snack where you can see the background colour is not visible. Is this a bug or am I doing something wrong?
In my app my background is a gradient image so I am trying to have the NavBar sit on top of it and be transparent.
import { Text } from 'react-native';
import { NativeRouter } from 'react-router-native';
import { Navigation, Card } from 'react-router-navigation';

export const App = () => (
  <View style={{ backgroundColor: 'red', flexGrow: 1 }}>
    <NativeRouter>
      <Navigation
        navBarStyle={{
          backgroundColor: 'transparent',
          borderBottomWidth: 0,
        }}
        cardStyle={{
          backgroundColor: 'transparent',
          shadowOpacity: 0,
        }}>
        <Card exact path="/" render={() => <Text>Home</Text>} />
        <Card path="/another" render={() => <Text>Another</Text>} />
      </Navigation>
    </NativeRouter>
  </View>
);

I've figure this out.... Adding the following to <Navigation /> solves it:
transitionConfig={() => ({
  containerStyle: {
    backgroundColor: 'transparent',
  },
})}


Invariant Violation: Failed to call into JavaScript module method AppRegistry.runApplication().

Description
Hi, just after adding a new package, a get a big error, and I can't start my application (it block on slapsh screen picture)
ERROR  [Reanimated] Couldn't determine the version of the native part of Reanimated. Did you forget to re-build the app after upgrading react-native-reanimated? If you use Expo Go, you must use the exact version which is bundled into Expo SDK.
ERROR  Error: Requiring module "node_modules/react-native-reanimated/src/index.ts", which threw an exception: TypeError: this.InnerNativeModule.makeShareableClone is not a function. (In 'this.InnerNativeModule.makeShareableClone(value, shouldPersistRemote)', 'this.InnerNativeModule.makeShareableClone' is undefined)
ERROR  Invariant Violation: Failed to call into JavaScript module method AppRegistry.runApplication(). Module has not been registered as callable. Registered callable JavaScript modules (n = 11): Systrace, JSTimers, HeapCapture, SamplingProfiler, RCTLog, RCTDeviceEventEmitter, RCTNativeAppEventEmitter, GlobalPerformanceLogger, JSDevSupportModule, HMRClient, RCTEventEmitter.
A frequent cause of the error is that the application entry file path is incorrect. This can also happen when the JS bundle is corrupt or there is an early initialization error when loading React Native.
ERROR  Invariant Violation: Failed to call into JavaScript module method AppRegistry.runApplication(). Module has not been registered as callable. Registered callable JavaScript modules (n = 11): Systrace, JSTimers, HeapCapture, SamplingProfiler, RCTLog, RCTDeviceEventEmitter, RCTNativeAppEventEmitter, GlobalPerformanceLogger, JSDevSupportModule, HMRClient, RCTEventEmitter.
A frequent cause of the error is that the application entry file path is incorrect. This can also happen when the JS bundle is corrupt or there is an early initialization error when loading React Native.
I don't understand the problem and I ignore how to solve it...
React Native Version
0.70.5
Output of npx react-native info
info Fetching system and libraries information...
System:
OS: Linux 5.15 Ubuntu 22.04.2 LTS 22.04.2 LTS (Jammy Jellyfish)
CPU: (8) x64 Intel(R) Core(TM) i7-1065G7 CPU @ 1.30GHz
Memory: 2.93 GB / 7.65 GB
Shell: 5.1.16 - /bin/bash
Binaries:
Node: 18.14.2 - ~/.nvm/versions/node/v18.14.2/bin/node
Yarn: 1.22.19 - /usr/bin/yarn
npm: 9.5.0 - ~/.nvm/versions/node/v18.14.2/bin/npm
Watchman: Not Found
SDKs:
Android SDK: Not Found
IDEs:
Android Studio: Not Found
Languages:
Java: Not Found
npmPackages:
@react-native-community/cli: Not Found
react: 18.1.0 => 18.1.0
react-native: 0.70.5 => 0.70.5
npmGlobalPackages:
react-native: Not Found
Steps to reproduce
I do this line of command to add a package :
yarn add react-native-reanimated react-native-safe-area-context
And after nothing works again, I can't load my app on expo
Snack, code example, screenshot, or link to a repository
This is my package.json :
{   "name": "testApp",   "version": "1.0.0",   "main": "node_modules/expo/AppEntry.js",   "scripts": {     "start": "expo start",     "android": "expo start --android",     "ios": "expo start --ios",     "web": "expo start --web"   },   "dependencies": {     "@emotion/react": "^11.10.6",     "@emotion/styled": "^11.10.6",     "@fortawesome/fontawesome-free-solid": "^5.0.13",     "@fortawesome/fontawesome-svg-core": "^6.3.0",     "@fortawesome/free-brands-svg-icons": "^6.3.0",     "@fortawesome/free-regular-svg-icons": "^6.3.0",     "@fortawesome/free-solid-svg-icons": "^6.3.0",     "@fortawesome/react-fontawesome": "^0.2.0",     "@fortawesome/react-native-fontawesome": "^0.3.0",     "@mui/material": "^5.11.12",     "@react-native-async-storage/async-storage": "^1.17.11",     "@react-native-community/datetimepicker": "6.5.2",     "@react-navigation/bottom-tabs": "^6.5.4",     "@react-navigation/native": "^6.1.3",     "@react-navigation/stack": "^6.3.12",     "@shopify/flash-list": "1.3.1",     "axios": "^1.3.4",     "eas-cli": "^0.60.0",     "expo": "~47.0.12",     "expo-linear-gradient": "^12.0.1",     "expo-status-bar": "~1.4.2",     "react": "18.1.0",     "react-dom": "18.1.0",     "react-native": "0.70.5",     "react-native-animated-dots-carousel": "^1.0.2",     "react-native-animated-pagination-dot": "^0.4.0",     "react-native-encrypted-storage": "^4.0.3",     "react-native-gesture-handler": "~2.8.0",     "react-native-map-clustering": "^3.4.2",     "react-native-maps": "^1.3.2",     "react-native-reanimated": "^3.0.1",     "react-native-reanimated-carousel": "^3.3.0",     "react-native-safe-area-context": "^4.5.0",     "react-native-screens": "~3.18.0",     "react-native-snap-carousel": "^3.9.1",     "react-native-svg": "13.4.0",     "react-native-ui-lib": "^6.31.0",     "react-native-vector-icons": "^9.2.0",     "react-native-web": "~0.18.9"   },   "devDependencies": {     "@babel/core": "^7.12.9",     "@types/lodash": "^4.14.191",     "@types/react": "~18.0.14",     "@types/react-native": "~0.70.6",     "@types/react-native-snap-carousel": "^3.8.5",     "@types/react-native-vector-icons": "^6.4.13",     "typescript": "^4.6.3"   },   "private": true }

ERROR [Reanimated] Couldn't determine the version of the native part of Reanimated

Hi,
Thanks for reporting this issue.
It seems like this issue is actually unrelated to React Native core but related to Reanimated. Please open this issue against this other repository:
https://github.com/software-mansion/react-native-reanimated/issues
I'll be closing this but feel free to reopen if the other repo points back to us as a issue in the core of React Native.


modify Events schema

Topic
WHAT DOES THIS PR DO?
As we move towards using our database more for device measurements (sensor readings), we had to modify the Events schema accordingly so that it focuses more on the Device rather than the Sensor during the storage of readings. A review of the main points this PR addresses follow:

[x] This PR makes modifications to the Event Schema so that it stores data with respect to a device and not a sensor.
[x] Also updates the API endpoint for adding Events, input is now an array that take in one or more device measurement objects
[x] Optimises the performance of the GET Events (measurements) endpoint.
[x] Adds optional query parameters (startTime, endTime, skip and limit) to the GET Events endpoint
[ ] Incorporate unit and integration tests accordingly
[ ] As a bonus, the PR also adds a script for inserting multiple measurements from an external API with time series data into the Events collection.

WHAT JIRA STORY/TASK IS RELATED TO THIS PR?

Running without Docker
Remember to start Redis server in a separate shell terminal of your shell, run redis-server for MAC?
cd AirQo-api/src/device-registry
npm install

npm run dev-mac for MAC OR npm run dev-pc for Windows
Using Docker in development ENV
cd AirQo-api/src/device-registry
docker build -t device-reg .
docker run -it -p 3000:3000 device-reg

WHICH ENDPOINTS SHOULD BE READY FOR TESTING?:
base url: http://localhost:3000/api/v1/devices

Add Values partial url [POST]: events/add?device={DEVICE NAME}&tenant={ORGANISATION}

[
    {
        "frequency": "minute",
        "time": "2021-02-17T05:40:29.774Z",
        "channelID": 233656
        "pm1": {
            "value": 232,
            "calibratedValue": 67,
            "uncertaintyValue": 39,
            "standardDeviationValue": 59
        },
        "pm2_5": {
            "value": 67,
            "calibratedValue": 67,
            "uncertaintyValue": 39,
            "standardDeviationValue": 59
        },
        "s2_pm2_5": {
            "value": 56,
            "calibratedValue": 67,
            "uncertaintyValue": 39,
            "standardDeviationValue": 59
        },
        "pm10": {
            "value": 27,
            "calibratedValue": 67,
            "uncertaintyValue": 39,
            "standardDeviationValue": 59
        },
        "s2_pm10": {
            "value": 22,
            "calibratedValue": 67,
            "uncertaintyValue": 39,
            "standardDeviationValue": 59
        },
        "no2": {
            "value": 45,
            "calibratedValue": 67,
            "uncertaintyValue": 39,
            "standardDeviationValue": 59
        },
        "battery": {
            "value": 27
        },
        "location": {
            "latitude": {
                "value": 73
            },
            "longitude": {
                "value": 39
            }
        },
        "altitude": {
            "value": 34
        },
        "speed": {
            "value": 83
        },
        "satellites": {
            "value": 49
        },
        "hdop": {
            "value": 62
        },
        "internalTemperature": {
            "value": 53
        },
        "externalTemperature": {
            "value": 20
        },
        "internalHumidity": {
            "value": 48
        },
        "externalHumidity": {
            "value": 92
        },
        "externalPressure": {
            "value": 42
        }
    }
]

Get values partial url [GET]: events?device={DEVICE_NAME}&tenant={ORGANISATION}&startTime={YEAR-MM-DD}&endTime={YEAR-MM-DD}&skip={NUMBER}&limit={NUMBER}

Please note that startTime, device,  limit, skip and endTime query parameters are all optional.
IS THERE ANY JENKINS CONSOLE LINK FOR CODE COVERAGE AND BUILD INFO?
N/A
ARE THERE ANY RELATED PRs?
Yes, related to this PR for refactoring the Analytics microservice
ANY SCREENSHOT(S)?
Yes, the resultant document in the Events collection after running either the single or bulk endpoint as described above.

The responses for the GET events endpoint (with and without the date query parameters)

Testing the endpoint for adding events to the collection

Getting empty array but I guess this due to the caching??


original version of PCBWay RGB Glow Tube DIY Clock

I bought my clock from PCBWay a little over a year ago. I see now that the "new version" of their clock requires an "app" to set the time, date and patterns. This was not the case when I bought mine. I am assuming that the clock I purchased probably does not have the hardware required to utilize your firmware, does it?

Hard to know. include a photo of the PCB?


spirv-fuzz: Handle decorations during module donation

The spirv-fuzz module donation pass does not currently add decorations of donated ids to the recipient module.
Some decorations would not be appropriate - e.g. if donation has transplanted uniform variables and types and demoted them to have storage class Private, it would not make sense to transplant descriptor set and binding decorations.
But transplanting e.g. Relaxed decorations would make sense.
To support the Relaxed case, and similar, we would need an "add decoration" transformation, and something in the fact manager to record that result ids have been donated and that it is thus OK to do things to those result ids that may change semantics.

Closing this in favour of an approach where we add decorations randomly to dead and live-safe code.


Unable to resolve module react-native-otp-inputs in latest 2.0.1

When uses version 1 it works fine but latest version 2.0.1 have following problem.
While trying to resolve module react-native-otp-inputs
from file
F:\abc\app\components\EmailOTPVerification.js,
the package
F:\abc\node_modules\react-native-otp-inputs\package.json was successfully found.
However, this package itself specifies a main module field that could not be resolved
(F:\abc\node_modules\react-native-otp-inputs\lib\index.tsx.
Indeed, none of these files exist:

F:\abc\node_modules\react-native-otp-inputs\lib\index.tsx(.native||.android.js|.native.js|.js|.android.json|.native.json|.json)

F:\abc\node_modules\react-native-otp-inputs\lib\index.tsx\index(.native||.android.js|.native.js|.js|.android.json|.native.json|.json)

@Abhishekgarg727 closing in favour of #18. Let's keep a track of this here.
I could not reproduce this one by my self. Send a reproduction in the mentioned issue.

can you upload your example working code project

@Abhishekgarg727 I've uploaded in mentioned issue

@dsznajder  i looking forward to fix this issue so i created sample project using react-native-cli and reproduce this issue , here is my git repo of source code https://github.com/Abhishekgarg727/react-native-otp-inputs-error

@Abhishekgarg727 Ok, as I see, this will be related to react-native version. Today I will check which version is correct to use newest version. Please downgrade to 2.0.0. I will deprecate the newest one and create v3.0.0 as it's kinda breaking change.

@dsznajder  i did not test 2.0.0 version yet, let me compile code again to see that problem persists or not. i  also want to know, is this count as contribution ?

@Abhishekgarg727 Ok, as I see, this will be related to react-native version. Today I will check which version is correct to use newest version. Please downgrade to 2.0.0. I will deprecate the newest one and create v3.0.0 as it's kinda breaking change.

i tested version 2.0.0 same problem persists


Removal of strings now provided with rails-i18n gem

The commit 9f4227629263b655de9d3508c09eaab7fa226922 added
rails-i18n containing translations for some of the strings
we use. With this gem in place, we can safely remove
the duplicate strings from our en.yml, since these are no longer
needed.
Fixes: #1313

Travis failure seems to be relevant
 1) ActionView::Helpers::NumberHelper #number_to_rails_method
     Failure/Error: helper.number_to_rails_method(1234).should            == '1.2.kilobytes'
       expected: "1.2.kilobytes"
            got: "1.205.kilobytes" (using ==)
     # ./spec/helpers/number_helper_spec.rb:114:in `block (2 levels) in <top (required)>'
  2) ActionView::Helpers::NumberHelper#number_to_human_size without options hash
     Failure/Error: helper.number_to_human_size(1234).should            == '1.2 KB'
       expected: "1.2 KB"
            got: "1.205 KB" (using ==)

Per @martinpovolny comment, this is where the 'precision' key comes in . . . we had it set to 1, so is there a way to override such default values?  If not, we may not be able to just globally remove these.


Need some help with mochad

Hi Nicholas, sorry for contacting you this way but I can't find any of your working contacts.
I need to add one more function to mochad: ability to toggle unit off then on with fixed or configurable delay. I'm running mochad on RPi with X10 to toggle my lights in house but really want to be able to restart cable modem and router remotely. So, I need (on server side) to turn specific unit off, wait for nn second, than turn it on. Number of seconds can be hardcoded or whatever 😊

Resolved by myself; no help from "open source community" was received (no one even replied yet). BTW, it's OK ;)
P.S. But probably I'll never share my fixes...

Resolved by myself; no help from "open source community" was received (no one even replied yet). BTW, it's OK ;)
P.S. But probably I'll never share my fixes...

I sent this reply via email but Github doesn't seem to have added it to the issue. Not sure why.

Sorry, I don't remember what mochad is!
I stopped using X10 quite a number of years ago and switched to LightWave RF.
But now I am going look into using Sonoff modules instead (uses Wifi).

I am glad you managed to resolve your problem yourself.

I sent this reply via email but Github doesn't seem to have added it to the issue. Not sure why.

Sorry, I don't remember what mochad is!
I stopped using X10 quite a number of years ago and switched to LightWave RF.
But now I am going look into using Sonoff modules instead (uses Wifi).

I am glad you managed to resolve your problem yourself.


[backport] Fix docs: cupy-cuda112 now on PyPI

Backport of #4957

[automatic post] Jenkins, test this please.

Jenkins CI test (for commit 79e4428fae12efe28285758178f10318cac1f2cc, target branch v9) succeeded!


保存后弹出完成通知这个可以最小化么?

保存后弹出通知导致操作的时候, 还要点击一下界面, 感觉挺麻烦的

After the project is successfully saved, can the "Save Complete" dialog be minimized?
He thinks it is very inconvenient to hide the "Save Complete" dialog manually after saving the project.

Jerry: I think he is talking about this:
![image](https://user-images.githubusercontent.com/12783829/105081487-39d52e80-5acd-11eb-8d4f-339bbe0308e9.png)

After the project is successfully saved, can the "Save Complete" dialog be minimized?
He thinks it is very inconvenient to hide the "Save Complete" dialog manually after saving the project.

Jerry: I think he is talking about this:
![image](https://user-images.githubusercontent.com/12783829/105081487-39d52e80-5acd-11eb-8d4f-339bbe0308e9.png)

I can add an optional action that minimizes the popup after finishing all encodings, but that's going to be disabled by default.
I need to make sure that user knows where the encoding is being displayed and when it is ready to open the file.

I can add an optional action that minimizes the popup after finishing all encodings, but that's going to be disabled by default.
I need to make sure that user knows where the encoding is being displayed and when it is ready to open the file.

Thanks for the translation @Jerrylum :)

Thanks for the translation @Jerrylum :)

later


Need cluster info CSV file details

Hi Team,

In 6th step of Reconfigure cluster section, file name mentioned is ClusterConfig.csv. is it mistake or we need to create another file with this name. Because in all other above steps everywhere filename is mentioned as Cluster-Config.csv.

In very first line of article "always on" is mentioned where page is for fail over cluster and these are two different concept with respect to SQL server.

Please reply on<EMAIL_ADDRESS>as I am referring this document for our migration and waiting for your response.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 8c9cb342-fc08-2630-9a2a-3467e9651189
Version Independent ID: 05766c8a-c353-d151-18a4-2f3048a75456
Content: Migrate failover cluster instance - SQL Server on Azure VMs
Content Source: articles/azure-sql/migration-guides/virtual-machines/sql-server-failover-cluster-instance-to-sql-on-azure-vm.md
Service: virtual-machines-sql
Sub-service: migration-guide
GitHub Login: @rahugup
Microsoft Alias: rahugup

@athosar50 Thank you for bringing this to our attention.  My initial thought is that it's the same file and this is just a typo but we'll reach out to the doc author to confirm.
First reference of Cluster-Config.csv is step 2:
https://docs.microsoft.com/en-us/azure/azure-sql/migration-guides/virtual-machines/sql-server-failover-cluster-instance-to-sql-on-azure-vm#prepare-source-machines

Retrieve cluster information: Run the Get-ClusterInfo.ps1 script on a cluster node to retrieve information on the cluster resources. The script will output the role name, resource name, IP, and probe port in the Cluster-Config.csv file. Use this CSV file to create and assign resource in Azure later in this article.

First/only reference of ClusterConfig.csv (step 6):
https://docs.microsoft.com/en-us/azure/azure-sql/migration-guides/virtual-machines/sql-server-failover-cluster-instance-to-sql-on-azure-vm#reconfigure-cluster

Copy the ClusterConfig.csv file and run the Update-ClusterConfig.ps1 script passing the CSV as a parameter. This will ensure the cluster resources are updated with the new configuration for the cluster to work in Azure.

@MashaMSFT Can you please help confirm on these 2 questions:

In 6th step of Reconfigure cluster section, file name mentioned is ClusterConfig.csv. is it mistake or we need to create another file with this name. Because in all other above steps everywhere filename is mentioned as Cluster-Config.csv.

In very first line of article "always on" is mentioned where page is for fail over cluster and these are two different concept with respect to SQL server.

Thanks for the feedback! I have assigned the issue to the content author to investigate further and update the document as appropriate.

Hi @athosar50 , thanks so much for contacting Microsoft and our apologies for the delay in response! Yes, the correct filename is Cluster-Config.csv. We've processed the change and it will be live next week. However, Always On is an umbrella term for two features of SQL Server, availability groups, and failover cluster instances. As such, Always On failover cluster instance is the correct full name of the feature. You can see the same name being used in the official Always On failover cluster instance documentation. I hope that helps! I'll go ahead and close out this Git Issue but please feel free to comment should you need anything further.
Thanks again and I hope you have a wonderful rest of your day!
Masha from the SQL Docs team
#please-close


Make notice dismissable

Actual behaviour
The notice overlays everything. Some people do not know they can click through it.
Expected behaviour
The notice has an close button to dismiss the overlay until the next request.
Checklist

[x] I have checked all issues and haven't found a similar report.
[x] I am using the latest stable version of this plugin.

@Yarisade already implemented it in the HEPTACOM WordPress variant to FroshEnvironmentNotice. Maybe she has a minute left to implement it here too.


Fix output format for txids in sendprogress command

Running a CLT...

------------
     Summary [ 794.964s] 336 tests run: 336 passed (112 slow), 30 skipped

git show -s
commit 725a59c85396bf8d34dce78d9324b3dd7bd81edc (HEAD, dorian/sendprogress-out-format)
Author: dorianvp<EMAIL_ADDRESS>Date:   Thu Nov 21 23:31:42 2024 -0300

    fix: use json format for tx ids

I"m running a CLT, in the meantime let's ship this to @juanky201271 for a manual test?

Thanks @dorianvp I believe that this PR fixed this issue: https://github.com/zingolabs/zingolib/issues/1469


[UX] Authorisation View

The authorisation view should be improved by providing

[ ] Filter to trim the result set

[ ] Export to certain file types (json for starters)

[ ] Prevent Browsers from autofilling input fields. With safari you don't see the result list anymore

:tada: This issue has been resolved in version 0.103.0 :tada:
The release is available on:

GitHub release
v0.103.0

Your semantic-release bot :package::rocket:


Add conversion to reverse format and back for ipv4

I noticed there already are functions to convert to and from the reverse form for v6 addresses. I also need this functionality for v4 addresses, so this adds the same kind of functions to the v4 addresses.

@leftshift love it, thanks! will cut a release shortly after I merge 👍

released in 8.1.0 🎉


AMBARI-24390. Filter services eligible for Ambari Single Sign-on Conf…

…iguration if Kerberos is required but not enabled (amagyar)
What changes were proposed in this pull request?
Some services require Kerberos to be enabled for SSO to work. For example, HDFS, Yarn, and Oozie. For this case, the metadata is enhanced allowing for the metadata to indicate whether Kerberos is required. These services should be filtered from Ambari CLI when setting up SSO if not eligible when Kerberos is not enabled.
How was this patch tested?
Ran
$ ambari-server setups-sso

and checked the eligible service list in different cases

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/3420/
Test PASSed.


Avoid using the tests folder for the file manipulation test

https://github.com/rust-lang/rust/pull/65278 r? @RalfJung

That was quick, wow! But why would that work? Now it expects the file in the working directory instead of one level down, which still isn't where the file actually is.

That works because the test creates the file, then writes to it, then reads from it and then it deletes the file. The first version just assumed the file was there but now we don't need that anymore.

Ah, that must be in the PR I didn't get to review yet. ;)

Heh, yeah that's exactly why. So, this would only pollute the build dir if the test fails, but hopefully travis will catch that issue even before someone decides to bump miri.

Maybe give the file a better name then in case it doesn't get cleaned up, like miri_test_fs.txt?

sure

:hourglass: Testing commit e9138ab4d63019a514fe5d531776fc1ecd37c307 with merge e10d9d32d79aaf5860d6f90dd6ba26a1cce8c6b0...

:sunny: Test successful - checks-travis, status-appveyor
Approved by: RalfJung
Pushing e10d9d32d79aaf5860d6f90dd6ba26a1cce8c6b0 to master...


Operator deletes the PVC on cluster deletion even when reclaim policy is set to retain

Hi All,
It appears when deleting a cluster, the operator cleans up all resources as expected, but also deletes the PVC even when the storage class reclaim policy is set to retain.
Do you think it would be better to delegate the PVC lifecycle back to the storage provisioner?
Many thanks

hi @simonmacklin
Can you provide a example of the storageclass and the cluster you were deploying?
Also, the reclaim policy affects the PVs not the PVC, the PVC will be deleted but the PV will remain, is that the case?
Turning into discussion since isn't an issue.
Regards!


Consider and implement the desired level of the Operator Maturity Model

Given the five phases of the Operator Maturity Model:

Basic Install
Seamless Upgrades
Full Lifecycle
Deep Insights
Auto Pilot

Consider all phases and implement the necessary tasks to achieve the desired maturity phase.

Currently, we're at level 2 (Seamless Upgrades), and I'm not sure we want to move up the ladder in the nearest future unless we have a requested from the community. I suggest closing this one for now.

We do not have a demand to use this module, let's close


investigate fetching depths from list of points from IFIM

we have set up a strategy to read using http range requests with gdal - @rgoss submitted the basic code for this last week.


rewrite healthcheck for DA light node

The current health check works well for a local devnet, but we will need to come up with a health check for the DA light node that verifies whether or not the node has synced all the way. If the node has not synced, user will run into an error fetching data at latest height, when it is not synced to that height: https://github.com/celestiaorg/celestia-node/issues/2151

Will be shifting responsibility to user, if using light node https://github.com/celestiaorg/docs/pull/690/commits/8cfd30bf740a10318feeebd0ddf236aa3a18bba7


Sharing specs across models -- DRY?

With the old ImageKit API, I was able to create a single spec module at the site root that I could share across all of my image models within a project. This allowed me to have standard image sizes to use in various layouts. All I'd have to do is have that model inherit ImageModel and provide the path to the spec file.
Now, it seems that for each model that I want these ImageSpecs, I'll have to recreate each of the specs as fields on each individual model. And if I want to add or change specs, I'd need to edit each and every model.
Am I missing something with how this new API works? Is there a way to specify specs without having to spell out all the properties each time?
One thought that I had was to create an abstract model mixin that contains the site-wide specs have each of the models with images inherit that mixin. But this would require being very careful that the imageField be named the same field name.
The other problem is working with third party apps. Previously, I could create a proxy model that would give a third party model the ImageModel behavior and attach the spec file to it. I suppose I could do something similar now, but again the issue of field names remains.

You should be able to define your specs anywhere you want and then use them as model properties.  For example:
# allmyspecs.py
from imagekit.models import ImageSpec
thumbnail = ImageSpec(...)
giant_image = ImageSpec(...)

And then in your models file:
from django.db import models
import allmyspecs

class MyModel(models.Model):
    some_image = models.ImageField(...)
    thumbnail = allmyspecs.thumbnail
    giant_image = allmyspecs.giant_image

class MyOtherModel(models.Model):
    some_other_image = models.ImageField(...)
    best_size = allmyspecs.thumbnail

I admit, though, I haven't tested this very extensively so if you have issues, let us know.
(And if you have times, it would be great if you could contribute some tests for this usage!)

Thanks. I will try this out.
Three things still stand out for me with this method...If I add a new spec to allmyspecs.py, I still have to go into every model and add it -- rather then the model just pulling in the modified spec file. Secondly, I would still need all of the original image fields to be name the same, correct? Thirdly, what about third party apps? Perhaps a feature request could be template tags or template filters that can implement defined specs?
Thanks for your reply and for the work on this project. If I can contribute at all I will -- although I'm more of a designer that can hack his way around Django than a developer ;)

Yeah, if you add new specs, you would have to add them to the model.

No; if you omit the image_field argument when you create your spec, it will look for an ImageField on your model. If your model defines more than one ImageField, it'll throw an error, though, so—while this is still more flexible than the original API (which only ever allowed one ImageField per ImageModel)—there is room for improvement.

Are you asking about adding specs for ImageFields defined on third party models? To the best of my recollection, this wasn't possible with the original IK since any time you wanted to use a spec, you'd have to extend the ImageModel class. With the new IK, it is technically possible using the contribute_to_class method of Django's model fields:
from imagekit.models import ImageSpec
from thirdpartyapp.models import TheModel
fancy_thumbnail = ImageSpec(...)
fancy_thumbnail.contribute_to_class(TheModel, 'thumbnail')
print TheModel.objects.all()[0].thumbnail.url

The template tag thing is something I had considered, but I just couldn't think of a way to do it that I liked. Because of how Django's template tags work, you'd have to include the property you want to access ("url", "width", etc.). For example:
{% specresult mymodel.image spec 'width' %}

Blegh. Seems better to just process the image in the view and pass the result to the template…maybe we could do some work to facilitate that.

Thanks. For #1, I think I can avoid the duplication of efforts by doing something like this:
# allmyspecs.py

from django.db import models
from imagekit.models import ImageSpec

thumbnailSpec = ImageSpec(...)
smallSpec = ImageSpec(...)
mediumSpec = ImageSpec(...)
...

class ImageSpecMixin(models.Model):
    thumbnail = thumbnailSpec
    small = smallSpec
    medium = mediumSpec

# models.py

from django.db import models
import allmyspecs import ImageSpecMixin

class MyModel(ImageSpecMixin):
    image = models.ImageField(...)
    ...

#2  --- Ok, good to know, that will make things easier.
#3 --- Filebrowser has an interesting implementation of image template tags:
{% version_object mymodel.image 'spec' as var %}

Then you can do:
<img src="{{ var }}" width="{{ var.width }}" height="{{ var.height }}"

You know, for #1 you might actually be able to do it without inheritance, using the contribute_to_class method I mentioned earlier. Something like this:
# allmyspecs.py
from django.db import models
from imagekit.models import ImageSpec

thumbnail_spec = ImageSpec(...)
small_spec = ImageSpec(...)
medium_spec = ImageSpec(...)
def add_specs(model_class):
    thumbnail_spec.contribute_to_class(model_class, 'thumbnail')
    small_spec.contribute_to_class(model_class, 'small')
    medium_spec.contribute_to_class(model_class, 'medium')

# (Anywhere)
from myapp.models import MyModel
from thirdparty.models import TheirModel
from allmyspecs import add_specs
add_specs(MyModel)
add_specs(TheirModel)

As for Filebrowser's implementation, I'm personally not a fan for a few reasons:

It shoehorns logic into the template that would probably be better off in code
Having a string represent a spec means there has to be some kind of registry and we have to worry about collisions between apps, etc.

Now that I'm thinking about it more, I think I probably would go with a Proxy model. (Sorry, I had completely forgotten you mentioned that in your original post!) It keeps all of your code nice and organized. And, I think the add_specs function I described above would accomplish exactly what you used to do by specifying a new specs module—that is: centralize your spec definitions. You could also probably address your field name concerns in add_specs. I can't think of anything you could do with the specs module that you wouldn't be able to do in an add_specs-like function.

You know, I think it's very rare that you'd need to do anything complicated enough to warrant changing the class definition at runtime by using contribute_to_class. In most cases, you could probably get away with a Proxy model, predefined specs, and maybe a mixin. For example:
thumbnail_spec = ImageSpec(...)
small_spec = ImageSpec(...)

class ImageSpecMixin(object):
    small = small_spec
    medium = ImageSpec(...)

class MyModelProxy(MyModel, ImageSpecMixin):
    thumbnail = thumbnail_spec
    large = ImageSpec(...)

    class Meta:
        proxy = True

class TheirModelProxy(TheirModel, ImageSpecMixin):
    tiny = thumbnail_spec

    class Meta:
        proxy = True

@dmeehan It seems like you've got a pretty DRY solution to your problem so I'm going to close this. Please reopen if I'm mistaken.
I think we talked about a lot of great stuff in this thread—especially the use of Proxy models to augment third-party models with image specs. If you're feeling up to it, it would be awesome if you could contribute a page to the docs talking about this stuff.
Thanks for your thoughtful posts and for helping me to think about these strategies a little more clearly!

I agree that this solution will work with ImageKit as it stands. However, I don't think it is my ideal solution to the issue of how to handle images in Django. I don't suggest that this is everyone's ideal, but I've thought a lot about this issue so I'll give my two cents.
I'd love to be able to decouple image versioning from my other apps as much as possible, in the most simple way. I think my ideal situation would be as follows:
-- Models only define an imageField (the data). They shouldn't (necessarily) need to know about what app I'm using to manipulate that image file. I should be able to plug any image manipulation app in without having to change my models.
-- Specs are defined in a centralized place and can be applied across different models. These specs can be accessed from different levels (in python code, explicitly attached to models, or through template tags). While I'd rather not have to attach these specs at the model level, there might be cases where this makes sense.
-- I'm thinking more and more that template tags are the right place to implement the specs. You said at one point that "It shoehorns logic into the template that would probably be better off in code". I don't think this is the case. The logic is still in the code -- the templatetag code. The template is simply telling the code "use X image and apply Y spec to it". Which spec to use is a presentational concern and so template tags seem like the right place for it. Other apps (easy_thumbnails, sorl_thumbnail) use templatetags, but I don't think they have the power of ImageKit's notion of processors and specs. It would be great to have both.
So, what I'd like to be able to do is:
-- plug imagekit into my project
-- create an imagespecs.py file at the root of my project that defines my processors and specs
-- apply those specs through a template tag
That way, if I decide for some reason to no longer to use image manipulation or to use another library, all I have to edit are my templates. Templates being the most custom and fluid part of Django, this seems appropriate.
As I stated earlier, I'm much more of a designer than a developer, so I don't know the limitations or problems with this kind of approach. However, I thought I would add these ideas to the discussion.

Models only define an imageField (the data). They shouldn't (necessarily) need to know about what app I'm using to manipulate that image file. I should be able to plug any image manipulation app in without having to change my models.

Alright, let's call this goal number 1: eliminating your apps' dependency on IK.

Specs are defined in a centralized place and can be applied across different models. These specs can be accessed from different levels (in python code, explicitly attached to models, or through template tags). While I'd rather not have to attach these specs at the model level, there might be cases where this makes sense.

This will be goal 2: centralizing spec definition.
Both of these goals are clearly good ones. I don't know why you keep saying you're "just a designer"—these are the kinds of concerns good developers have! So let's get down to hashing out a solution…

I'm thinking more and more that template tags are the right place to implement the specs. You said at one point that "It shoehorns logic into the template that would probably be better off in code". I don't think this is the case. The logic is still in the code -- the templatetag code.

I think you misunderstood me here. This comment was in regards to FileBrowser's implementation—specifically the assignment of spec to a template variable. In my opinion, this kind of assignment shouldn't be done in the template; it feels like procedural code translated into the template language.
So I'm not initially predisposed against a template tag approach. Indeed, it seems the obvious course to take given our goal of decoupling spec definition from the model definition. But it does have some issues that make it a very hairy solution:
Firstly, the implementation of the image creation is now hard-coded into all of your templates. If you decide that maybe your thumbnail shouldn't be automatically generated, but rather uploaded by the user, you need to update every template that uses that thumbnail.
Secondly, there's the issue of designing the tag itself. As I hinted at when discussing FileBrowser's tag, Django's templating language makes it very difficult to come up with something that doesn't suck. The basic difficulty being that we're dealing with an object (the file-like ImageSpecFile) and we need to be able to access all of its properties. So far, I've been unable to think of a satisfactory alternative to a FileBrowseresque approach:
{% image_spec mymodel.original_image 'thumbnail' as thumbnail %}

I really really dislike the assignment in the template. But I guess some people might disagree, so we'll count it as a strike against the approach instead of a disqualifier. Unfortunately, there are a few other really nasty things going on. For one, we're putting the source image field ("original_image") in every template. That's a lot of duplication and a potential problem if it ever changes, but we could avoid the issue by moving the field name into the spec definition (thereby centralizing it) and then we'd be left with something like this:
{% image_spec mymodel 'thumbnail' as thumbnail %}

However, the source image fields of mymodel and yourmodel don't necessarily  share the same name, so this solution would seem to be off the table (but keep it in your mind—I'll be referring back to it later). For now, it looks as though we're stuck with repetition of the source image field in every template. Either way, the FB template tag approach is a lot more verbose than our current solution (again, not necessarily a disqualifier in itself). The problem is only compounded when you consider included templates, each of which would need to use the template tag again to guarantee the spec was available without making assumptions about its includer.
Moving past the design of the tag, let's look at the definition of the spec itself. It's an absolute requirement that specs be definable on an app-level (otherwise they wouldn't be distributable), so let's work under the assumption that your proposed imagespecs.py can be located in the app folder in addition to the project folder. What does it look like? Well, it has to define a spec and register it with a id so that it can be looked up by the template tag. Immediately we hit a wall: what if two separate apps register a spec with the same id (e.g. "thumbnail")? Apparently, we would need some kind of scoping system where the specs defined in app A don't apply to app B, and a project-level imagespecs.py is used as a fallback. I guess we'd also have to provide a way to override an app's specs, but we're still not out of the woods. How could you choose to selectively override only app A's "thumbnail" spec definition? What it boils down to is this: our registration would also have to allow us to target specific applications; something like this:
register(spec_id, spec, app=None)

Keep in mind that I'm not proposing a specific syntax here, I'm just trying to make concrete the things that any solution would need to include.
Still, this doesn't give you much granularity of control. The fact is that specs shouldn't really be specific to applications, but rather to fields; it's entirely possible (even likely) that you may decide that the thumbnail version of the Dog.photo image should be different than the thumbnail version of the Cat.photo image, even if both the Dog and Cat models are defined in an "animals" app. (Say you want to overlay the text "ruff" on the dog, but "meow" on the cat.) Given where our proposal stands, that would mean creating separate "dog_thumbnail" and "cat_thumbnail" specs and overriding every single template in which the "thumbnail" spec was used to replace it with the appropriate dog or cat version. Clearly, this is unacceptable. We have to revise our registration to allow us to target specific fields. Something like this:
register(spec_id, spec, app=None, model=None, source_field=None)

Again, I'm not proposing a specific syntax—these fields could be provided in a few different ways. I'm just trying to clarify the pieces that any solution requires at a bare minimum.
We might reflect that by including the model and source_field in our registration, we're very close to being able to use the revised FileBrowser template tag syntax (that I said I'd be referring back to). After all, the problem was that the syntax didn't account for models having different source fields, but if we're willing to stipulate that spec_id corresponds to a bound source_field, then we've eliminated that issue.
At this point, something surprising strikes me: we've just recreated the current, field-based system! The formulation of spec_id corresponding to a specific field of a model is exactly what we accomplish by assigning ImageSpecs as model fields: the only thing that's changed with the template tag solution is the method of lookup (through the registry instead of as an attribute). The registration method also use the same "pieces": identifier (attribute name), spec, app, model, and source field. All we've done is tweaked the syntax. Even if we're not willing to accept the final leap (the stipulation that spec_id corresponds to a bound source_field and not just a spec), I think it's important to recognize how close we've come to full circle and how many concessions we had to make to get there (verbosity, increased repetition, greater code complexity). In other words, by the time we've smoothed out the issues with a template tag approach, we've got basically the exact same system with a lot more mess.
Of course, that mess may be justified since we've solved our two initial goals. So the question shifts to whether it's possible to address those goals without incurring the mess that the template tag approach incurs. I think the answer is yes. By using the contribute_to_class approach, you can both completely eliminate all dependencies on IK from your apps and centralize your spec definitions. In fact, what you'd like to do is almost exactly what you can do:

So, what I'd like to be able to do is:
-- plug imagekit into my project
-- create an imagespecs.py file at the root of my project that defines my processors and specs
-- apply those specs through a template tag

The only thing the contribute_to_class approach doesn't do is use a template tag, but that's not really one of our end-goals anyway and I think I've provided a lot of reasons why the template tag approach ends up being problematic. Just to make it more tangible, here's an example that adds thumbnail specs (some the same, some not) to several different apps from outside of all of them:
# imagespecs.py (in root of project)
from django.db import models
from imagekit.models import ImageSpec
from thirdpartyapp.models import Dog, Cat, Fish
from myapp.models import Profile

thumbnail_spec = ImageSpec(...)
dog_thumbnail_spec = ImageSpec(...)
cat_thumbnail_spec = ImageSpec(...)

thumbnail_spec.contribute_to_class(Profile, 'thumbnail')
thumbnail_spec.contribute_to_class(Fish, 'thumbnail')
dog_thumbnail_spec.contribute_to_class(Dog, 'thumbnail')
cat_thumbnail_spec.contribute_to_class(Cat, 'thumbnail')

You'd just have to make sure that this got imported somewhere. Then, in your templates, you'd access the specs as normal attributes:

A lot cleaner and a lot less complicated if you ask me.
Hopefully it comes through that I've given this a lot of thought. I agree that the template tag seems intuitively to be the right approach and when I dug into the problem it was with the assumption that I'd be writing something in that vein. But, having continued down that road a little ways, it became apparent that that approach had a lot of ugly spots, and the field approach is capable of addressing our concerns pretty elegantly.

No downsides performance-wise. In fact, it's called by Django's model's metaclass internally when you define a field "normally" (i.e. in the model class definition).
"changing the class definition at runtime" was a horribly chosen phrase; sorry. What I should have said was "changing the class definition outside of the class declaration." As for whether it's generally a best practice: no.
Naturally, the class declaration is where people are going to look to figure out how your class works, so in general that's where you should be adding attributes. If you can't—or don't want to—the next best place would be a proxy model, since you would still be adding attributes in a class definition and it would be clear where you were using instances of that class (and therefore on which objects those properties were accessible). Since Django views are already coupled to models, this is almost always good enough. However, if you want to be able to just update templates, this method isn't going to work and we only have two alternatives: monkey patching the model or creating a registry and performing a lookup in the template itself (a la Filebrowser).
As I've said, I think that these spec images actually are part of the model; changing how they're created—for example, having the user upload them instead of generating them programmatically—shouldn't change the way they're accessed. Add this to an unwillingness (or inability) to change models.py, and what we're saying is that we want to monkeypatch the model. That just means that you're going to have to rely on convention and documentation to indicate where the attributes are added.
Incidentally, using add_to_class is probably a little better than contribute_to_class since it gives the model class a chance to do some extra bookkeeping (though normally there's none to do).  So instead of
thumbnail_spec.contribute_to_class(Profile, 'thumbnail')

do:
Profile.add_to_class('thumbnail', thumbnail_spec)

Sorry I didn't mention this earlier.