Time of Day Not Followed

General troubleshooting tips
Before logging an issue please check that:

[x] You have the latest version installed (There may be a beta version that fixes your issue), see here for the latest releases and bug fixes: https://github.com/JohnCoates/Aerial/releases
[x] Your issue isn't already mentioned in our issues. You may find a workaround there or a similar request already made.
[x] Your problem isn't mentioned in the troubleshooting page.

If none of this fixes your issue, tell us about the problem you are experiencing or the feature you'd like to request.
Required information
In order to help us sort your issue, we ask that you provide the following information:

[x] Mac model: MacBook Pro (13-inch, 2020, Four Thunderbolt 3 ports)
[x] macOS version: 11.0.1 (20B50)
[x] Monitor setup: Dual spanned

If appropriate, please enable Debug mode and Log to disk in Advanced tab and replicate your bug, then attach the AerialLog.txt file (You can access this file through the Advanced tab).
2020-12-02 09:27:03.081 : avInit .saver (0.0, 0.0, 1440.0, 900.0) p: false o: false
2020-12-02 09:27:03.083 : Time Management initialized
2020-12-02 09:27:03.084 : Starting Location initialization
2020-12-02 09:27:03.084 : Location services enabled
2020-12-02 09:27:03.084 : <AerialView: 0x7fcb6470ce20> AerialView setup init (V2.2.1) preview: false
2020-12-02 09:27:03.085 : Running Version 11.0.1 (Build 20B50)
2020-12-02 09:27:03.085 : Display Detection initialized
2020-12-02 09:27:03.086 : ***Display Detection***
2020-12-02 09:27:03.086 : 2 display(s) detected
2020-12-02 09:27:03.086 : 1cm = 50.393701544239036 points
2020-12-02 09:27:03.087 : npass: dict [__C.NSDeviceDescriptionKey(_rawValue: NSDeviceResolution): NSSize: {144, 144}, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceBitsPerSample): 8, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceColorSpaceName): NSCalibratedRGBColorSpace, __C.NSDeviceDescriptionKey(_rawValue: NSScreenNumber): 69734208, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceIsScreen): YES, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceSize): NSSize: {1440, 900}]
2020-12-02 09:27:03.087 :        bottomLeftFrame (0.0, 0.0, 1440.0, 900.0)
2020-12-02 09:27:03.087 : npass: dict [__C.NSDeviceDescriptionKey(_rawValue: NSDeviceResolution): NSSize: {72, 72}, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceBitsPerSample): 8, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceColorSpaceName): NSCalibratedRGBColorSpace, __C.NSDeviceDescriptionKey(_rawValue: NSScreenNumber): 441001153, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceIsScreen): YES, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceSize): NSSize: {1920, 1080}]
2020-12-02 09:27:03.087 :        bottomLeftFrame (-1920.0, 496.0, 1920.0, 1080.0)
2020-12-02 09:27:03.088 : src orig : (0.0, 0.0)
2020-12-02 09:27:03.088 : left 1.0 below 0.0
2020-12-02 09:27:03.088 : src orig : (-1920.0, 496.0)
2020-12-02 09:27:03.088 : left 0.0 below 0.0
2020-12-02 09:27:03.089 : [id=69734208, width=1440, height=900, bottomLeftFrame=(0.0, 0.0, 1440.0, 900.0), topRightCorner=(1440.0, 900.0), isMain=true, backingScaleFactor=2.0]
2020-12-02 09:27:03.089 : [id=441001153, width=1920, height=1080, bottomLeftFrame=(-1920.0, 496.0, 1920.0, 1080.0), topRightCorner=(0.0, 1576.0), isMain=false, backingScaleFactor=1.0]
2020-12-02 09:27:03.089 : (-1920.0, 0.0, 3360.0, 1576.0)
2020-12-02 09:27:03.089 : ***Display Detection Done***
2020-12-02 09:27:03.090 : Real screen count : 2
2020-12-02 09:27:03.090 : Using : Optional([id=69734208, width=1440, height=900, bottomLeftFrame=(0.0, 0.0, 1440.0, 900.0), topRightCorner=(1440.0, 900.0), isMain=true, backingScaleFactor=2.0])
2020-12-02 09:27:03.090 : <AerialView: 0x7fcb6470ce20> setting up player layer with bounds/frame: (0.0, 0.0, 1440.0, 900.0) / (0.0, 0.0, 1440.0, 900.0)
2020-12-02 09:27:03.092 : allDisplays
2020-12-02 09:27:03.092 : mainOnly
2020-12-02 09:27:03.093 : mainOnly
2020-12-02 09:27:03.093 : Using dot workaround for video driver corruption
2020-12-02 09:27:03.093 : tvOS 13 is enabled, cached and up to date
2020-12-02 09:27:03.094 : tvOS 12 is enabled, cached and up to date
2020-12-02 09:27:03.094 : tvOS 11 is enabled, cached and up to date
2020-12-02 09:27:03.094 : tvOS 10 is enabled, cached and up to date
2020-12-02 09:27:03.361 : <AerialView: 0x7fcb6470ce20> backing change 2.0 isDisabled: false frame: (0.0, 0.0, 1440.0, 900.0) preview: false
2020-12-02 09:27:03.362 : LMauth status change : 3
2020-12-02 09:27:03.362 : Location coordinate : CLLocationCoordinate2D(latitude: [redacted], longitude: [redacted])
2020-12-02 09:27:03.363 : Location found [redacted] [redacted]
2020-12-02 09:27:03.365 : Refreshing video list
2020-12-02 09:27:03.365 : Poi Strings Provider initialized
2020-12-02 09:27:03.365 : Preferred languages : ["en-PH", "fil-PH"]
2020-12-02 09:27:03.366 : Best matched language : en
2020-12-02 09:27:03.366 : Defaulting to bundle
2020-12-02 09:27:03.366 : path : /Users/[redacted]/Library/Screen Savers/Aerial.saver/Contents/Resources/en.json
2020-12-02 09:27:03.366 : Community JSON : 99 entries
2020-12-02 09:27:03.367 : source : tvOS 13 contains 90 new videos (total 90)
2020-12-02 09:27:03.367 : source : tvOS 12 contains 0 new videos (total 90)
2020-12-02 09:27:03.367 : source : tvOS 11 contains 0 new videos (total 90)
2020-12-02 09:27:03.368 : source : tvOS 10 contains 9 new videos (total 99)
2020-12-02 09:27:03.368 : <AerialView: 0x7fcb6470ce20> pnv
2020-12-02 09:27:03.368 : <AerialView: 0x7fcb6470ce20> Setting player for all player layers in [<AerialView: 0x7fcb6470ce20>]
2020-12-02 09:27:03.369 : remaining in playlist : 0
2020-12-02 09:27:03.369 : generate playlist
2020-12-02 09:27:03.369 : playlist count: 91
2020-12-02 09:27:03.369 : Final count : 6
2020-12-02 09:27:03.370 : initvideocache
2020-12-02 09:27:03.370 : cached video file with length: 196041941
2020-12-02 09:27:03.370 : AssetLoaderDelegate deinit
2020-12-02 09:27:03.371 : <AerialView: 0x7fcb6470ce20> playing video (OFFLINE MODE) : file:///Users/[redacted]/Library/Containers/com.apple.ScreenSaver.Engine.legacyScreenSaver/Data/Library/Application%20Support/Aerial/Cache/comp_GMT306_139NC_139J_3066_CALI_TO_VEGAS_v08_HDR_PS_20180824_HDR_4K_HEVC.mov
2020-12-02 09:27:03.371 : <AerialView: 0x7fcb6470ce20> observing current item <AVPlayerItem: 0x600000fb1ea0, asset = <AVURLAsset: 0x600000df93c0, URL = file:///Users/[redacted]/Library/Containers/com.apple.ScreenSaver.Engine.legacyScreenSaver/Data/Library/Application%20Support/Aerial/Cache/comp_GMT306_139NC_139J_3066_CALI_TO_VEGAS_v08_HDR_PS_20180824_HDR_4K_HEVC.mov>>
2020-12-02 09:27:03.371 : <AerialView: 0x7fcb6470ce20> observeValue Optional("readyForDisplay") false
2020-12-02 09:27:03.375 : avInit .saver (-1920.0, 496.0, 1920.0, 1080.0) p: false o: false
2020-12-02 09:27:03.375 : <AerialView: 0x7fcb64414780> AerialView setup init (V2.2.1) preview: false
2020-12-02 09:27:03.376 : Running Version 11.0.1 (Build 20B50)
2020-12-02 09:27:03.376 : Real screen count : 2
2020-12-02 09:27:03.377 : Using : Optional([id=441001153, width=1920, height=1080, bottomLeftFrame=(-1920.0, 496.0, 1920.0, 1080.0), topRightCorner=(0.0, 1576.0), isMain=false, backingScaleFactor=1.0])
2020-12-02 09:27:03.377 : <AerialView: 0x7fcb64414780> setting up player layer with bounds/frame: (0.0, 0.0, 1920.0, 1080.0) / (0.0, 0.0, 1920.0, 1080.0)
2020-12-02 09:27:03.377 : allDisplays
2020-12-02 09:27:03.378 : mainOnly
2020-12-02 09:27:03.378 : mainOnly
2020-12-02 09:27:03.379 : Using dot workaround for video driver corruption
2020-12-02 09:27:03.379 : <AerialView: 0x7fcb64414780> backing change 1.0 isDisabled: false frame: (0.0, 0.0, 1920.0, 1080.0) preview: false
2020-12-02 09:27:03.590 : <AerialView: 0x7fcb6470ce20> observeValue Optional("readyForDisplay") true
2020-12-02 09:27:03.591 : start playback: (0.0, 0.0, 1440.0, 900.0) (0.0, 0.0, 1440.0, 900.0)
2020-12-02 09:27:03.591 : <AerialView: 0x7fcb6470ce20> startAnimation frame (0.0, 0.0, 1440.0, 900.0) bounds (0.0, 0.0, 1440.0, 900.0)
2020-12-02 09:27:03.876 : <AerialView: 0x7fcb64414780> startAnimation frame (0.0, 0.0, 1920.0, 1080.0) bounds (0.0, 0.0, 1920.0, 1080.0)
2020-12-02 09:28:38.235 : Location using cached data
2020-12-02 09:28:52.348 : Location using cached data

Description of issue / Feature request
The Time setting is "Use my location". If I click Test, it shows a success message that Aerial can access my location. However, the video being played doesn't match the current time. For example, it's already 9 AM but it's playing nighttime videos.

Hey @zgosalvez
Thanks for the report, I will have to put up a beta with better logging as I'm not quite sure what's happening here. Cali to vegas is indeed a night video.
A few things if you could check :

Can you go into the time tab and tell me what you have in your bar at the bottom ? Here's mine to give you an example, at 9:27, I'm still in "sunrise" based on my settings :

I'm asking because in some circumstances (you don't have any sunrise video cached), Aerial will fallback to something else.

Second, what do you have for Cache settings ? How large is your cache limit, and how much do you have downloaded ?

Based on that I'll put a new beta up today to see if I can narrow it down a bit. Thanks

@zgosalvez please try 2.2.2beta1, can you launch it once in full screen, and show me the log ? Thanks !
https://github.com/JohnCoates/Aerial/releases/download/v2.2.2beta1/Aerial.saver.zip

@glouel, thank you for debugging this with me.

Time settings

Cache settings

2.2.2beta1 log

2020-12-03 08:52:25.471 : avInit .saver (0.0, 0.0, 1440.0, 900.0) p: false o: false
2020-12-03 08:52:25.472 : Time Management initialized
2020-12-03 08:52:25.472 : Starting Location initialization
2020-12-03 08:52:25.473 : Location services enabled
2020-12-03 08:52:25.473 : <AerialView: 0x7fe394c09cc0> AerialView setup init (V2.2.2beta1) preview: false
2020-12-03 08:52:25.473 : Running Version 11.0.1 (Build 20B50)
2020-12-03 08:52:25.473 : Display Detection initialized
2020-12-03 08:52:25.474 : ***Display Detection***
2020-12-03 08:52:25.474 : 1 display(s) detected
2020-12-03 08:52:25.474 : 1cm = 50.393701544239036 points
2020-12-03 08:52:25.474 : npass: dict [__C.NSDeviceDescriptionKey(_rawValue: NSScreenNumber): 69734208, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceResolution): NSSize: {144, 144}, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceSize): NSSize: {1440, 900}, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceBitsPerSample): 8, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceIsScreen): YES, __C.NSDeviceDescriptionKey(_rawValue: NSDeviceColorSpaceName): NSCalibratedRGBColorSpace]
2020-12-03 08:52:25.475 :        bottomLeftFrame (0.0, 0.0, 1440.0, 900.0)
2020-12-03 08:52:25.475 : src orig : (0.0, 0.0)
2020-12-03 08:52:25.475 : left 0.0 below 0.0
2020-12-03 08:52:25.475 : [id=69734208, width=1440, height=900, bottomLeftFrame=(0.0, 0.0, 1440.0, 900.0), topRightCorner=(1440.0, 900.0), isMain=true, backingScaleFactor=2.0]
2020-12-03 08:52:25.475 : (0.0, 0.0, 1440.0, 900.0)
2020-12-03 08:52:25.476 : ***Display Detection Done***
2020-12-03 08:52:25.476 : Real screen count : 1
2020-12-03 08:52:25.476 : Using : Optional([id=69734208, width=1440, height=900, bottomLeftFrame=(0.0, 0.0, 1440.0, 900.0), topRightCorner=(1440.0, 900.0), isMain=true, backingScaleFactor=2.0])
2020-12-03 08:52:25.476 : <AerialView: 0x7fe394c09cc0> setting up player layer with bounds/frame: (0.0, 0.0, 1440.0, 900.0) / (0.0, 0.0, 1440.0, 900.0)
2020-12-03 08:52:25.478 : allDisplays
2020-12-03 08:52:25.478 : mainOnly
2020-12-03 08:52:25.478 : mainOnly
2020-12-03 08:52:25.479 : Using dot workaround for video driver corruption
2020-12-03 08:52:25.479 : tvOS 13 is enabled, cached and up to date
2020-12-03 08:52:25.479 : tvOS 12 is enabled, cached and up to date
2020-12-03 08:52:25.479 : tvOS 11 is enabled, cached and up to date
2020-12-03 08:52:25.480 : tvOS 10 is enabled, cached and up to date
2020-12-03 08:52:25.779 : <AerialView: 0x7fe394c09cc0> backing change 2.0 isDisabled: false frame: (0.0, 0.0, 1440.0, 900.0) preview: false
2020-12-03 08:52:25.780 : LMauth status change : 3
2020-12-03 08:52:25.780 : Location coordinate : CLLocationCoordinate2D(latitude: [redacted], longitude: [redacted])
2020-12-03 08:52:25.781 : Location found [redacted] [redacted]
2020-12-03 08:52:25.783 : Refreshing video list
2020-12-03 08:52:25.783 : Poi Strings Provider initialized
2020-12-03 08:52:25.783 : Preferred languages : ["en-PH", "fil-PH"]
2020-12-03 08:52:25.783 : Best matched language : en
2020-12-03 08:52:25.783 : Defaulting to bundle
2020-12-03 08:52:25.784 : path : /Users/[redacted]/Library/Screen Savers/Aerial.saver/Contents/Resources/en.json
2020-12-03 08:52:25.784 : Community JSON : 99 entries
2020-12-03 08:52:25.784 : source : tvOS 13 contains 90 new videos (total 90)
2020-12-03 08:52:25.784 : source : tvOS 12 contains 0 new videos (total 90)
2020-12-03 08:52:25.785 : source : tvOS 11 contains 0 new videos (total 90)
2020-12-03 08:52:25.785 : source : tvOS 10 contains 9 new videos (total 99)
2020-12-03 08:52:25.785 : <AerialView: 0x7fe394c09cc0> pnv
2020-12-03 08:52:25.785 : <AerialView: 0x7fe394c09cc0> Setting player for all player layers in [<AerialView: 0x7fe394c09cc0>]
2020-12-03 08:52:25.785 : Dark Mode override
2020-12-03 08:52:25.786 : remaining in playlist : 0
2020-12-03 08:52:25.786 : generate playlist
2020-12-03 08:52:25.786 : Playlist raw count: 91 raw cached count 25 isRestricted: true restrictedTo: night
2020-12-03 08:52:25.786 : Final count : 6
2020-12-03 08:52:25.787 : initvideocache
2020-12-03 08:52:25.787 : cached video file with length: 369968186
2020-12-03 08:52:25.787 : AssetLoaderDelegate deinit
2020-12-03 08:52:25.787 : <AerialView: 0x7fe394c09cc0> playing video (OFFLINE MODE) : file:///Users/[redacted]/Library/Containers/com.apple.ScreenSaver.Engine.legacyScreenSaver/Data/Library/Application%20Support/Aerial/Cache/comp_GMT060_117NC_363D_1034_AUSTRALIA_v35_HDR_PS_FINAL_20180831_HDR_4K_HEVC.mov
2020-12-03 08:52:25.788 : <AerialView: 0x7fe394c09cc0> observing current item <AVPlayerItem: 0x600003945d90, asset = <AVURLAsset: 0x600003b75900, URL = file:///Users/[redacted]/Library/Containers/com.apple.ScreenSaver.Engine.legacyScreenSaver/Data/Library/Application%20Support/Aerial/Cache/comp_GMT060_117NC_363D_1034_AUSTRALIA_v35_HDR_PS_FINAL_20180831_HDR_4K_HEVC.mov>>
2020-12-03 08:52:25.788 : <AerialView: 0x7fe394c09cc0> observeValue Optional("readyForDisplay") false
2020-12-03 08:52:25.978 : <AerialView: 0x7fe394c09cc0> startAnimation frame (0.0, 0.0, 1440.0, 900.0) bounds (0.0, 0.0, 1440.0, 900.0)
2020-12-03 08:52:26.004 : <AerialView: 0x7fe394c09cc0> observeValue Optional("readyForDisplay") true
2020-12-03 08:52:26.005 : start playback: (0.0, 0.0, 1440.0, 900.0) (0.0, 0.0, 1440.0, 900.0)

@glouel, I think the "Show only night videos in Dark Mode". I tried turning it off and so far the screensaver is playing day time videos. Note that System Preferences > General > Appearance is set to Auto. I noticed this same behavior before with SourceTree's theme when it relies on the system appearance. SourceTree falsely detects dark mode when the system apperance is set to Auto.

This is super interesting @zgosalvez and might solve another issue that was reported to me, I will look into how my code works when Appearance is set to Auto and see if there's something here. Thanks a whole lot I'll get back to you.

@zgosalvez this should be fixed in 2.2.3 now ! Thanks again for the report!

@glouel, was my pleasure. Thanks for the quick fix!


long-running fiber performance loss since 1.14.0

require "wait_group"

def collatz(seed : Int64)
  steps = 0_i64

  while seed > 1
    while seed % 2 == 0
      steps &+= 1
      seed //= 2
    end

    if seed > 1
      steps &+= 1
      seed = seed &* 3 &+ 1
    end
  end

  steps
end

def calculate(total_seeds, batch_size, worker_size, batches)
  channel = Channel({Int64, Int64}).new(batches + 1)
  wg = WaitGroup.new(worker_size)

  p! batch_size
  p! batches

  worker_size.times do |i|
    spawn(name: "WORKER-#{i}") do
      while r = channel.receive?
        (r[0]...r[1]).each do |seed|
          steps = collatz(seed)

          if seed % 1_000_000 == 0
            print "Seed: #{seed} Steps: #{steps}\r"
          end
        end
      end
      wg.done
    end
  end

  start = Time.measure do
    r0 = 0_i64

    batches.times do
      r1 = r0 &+ batch_size
      channel.send({r0, r1})
      r0 = r1
    end

    if total_seeds - batch_size &* batches > 0
      channel.send({r0, total_seeds})
    end

    channel.close
    wg.wait
  end

  puts "\ncollatz took: #{start}"
end

# ---------------- common part ----------------

total_seeds = 1_000_000_000_i64
worker_size = ENV.fetch("CRYSTAL_WORKERS").to_i

p! total_seeds
p! worker_size

puts "-"*100

batch_size = 1000_i64
batches = (total_seeds // batch_size).to_i32
calculate(total_seeds, batch_size, worker_size, batches)

puts "-"*100

batch_size = 2000_i64
batches = (total_seeds // batch_size).to_i32
calculate(total_seeds, batch_size, worker_size, batches)

puts "-"*100

batch_size = 5000_i64
batches = (total_seeds // batch_size).to_i32
calculate(total_seeds, batch_size, worker_size, batches)

puts "-"*100

batch_size = 500_i64
batches = (total_seeds // batch_size).to_i32
calculate(total_seeds, batch_size, worker_size, batches)

puts "-"*100

batch_size = 100_i64
batches = (total_seeds // batch_size).to_i32
calculate(total_seeds, batch_size, worker_size, batches)

puts "-"*100

batches = worker_size
batch_size = (total_seeds // batches).to_i32
calculate(total_seeds, batch_size, worker_size, batches)

You can run  it with: CRYSTAL_WORKERS=16 crystal run --release -Dpreview_mt 1.cr
Following is the result when built with 1.13.2
total_seeds # =><PHONE_NUMBER>
worker_size # => 16
----------------------------------------------------------------------------------------------------
batch_size # => 1000
batches # => 1000000
Seed: 999000000 Steps: 162
collatz took: 00:00:12.577703068
----------------------------------------------------------------------------------------------------
batch_size # => 2000
batches # => 500000
Seed: 999000000 Steps: 162
collatz took: 00:00:12.815965558
----------------------------------------------------------------------------------------------------
batch_size # => 5000
batches # => 200000
Seed: 999000000 Steps: 162
collatz took: 00:00:12.833879590
----------------------------------------------------------------------------------------------------
batch_size # => 500
batches # => 2000000
Seed: 999000000 Steps: 162
collatz took: 00:00:12.944712438
----------------------------------------------------------------------------------------------------
batch_size # => 100
batches # => 10000000
Seed: 999000000 Steps: 162
collatz took: 00:00:13.189403820
----------------------------------------------------------------------------------------------------
batch_size # => 62500000
batches # => 16
Seed: 937000000 Steps: 224
collatz took: 00:00:14.090417122

Following is the result when built with 1.14.0
 ╰──➤ $ ./1
total_seeds # =><PHONE_NUMBER>
worker_size # => 16
----------------------------------------------------------------------------------------------------
batch_size # => 1000
batches # => 1000000
Seed: 999000000 Steps: 162
collatz took: 00:00:13.265471911
----------------------------------------------------------------------------------------------------
batch_size # => 2000
batches # => 500000
Seed: 999000000 Steps: 162
collatz took: 00:00:13.537970427
----------------------------------------------------------------------------------------------------
batch_size # => 5000
batches # => 200000
Seed: 999000000 Steps: 162
collatz took: 00:00:13.500822759
----------------------------------------------------------------------------------------------------
batch_size # => 500
batches # => 2000000
Seed: 999000000 Steps: 162
collatz took: 00:00:13.484737884
----------------------------------------------------------------------------------------------------
batch_size # => 100
batches # => 10000000
Seed: 999000000 Steps: 162
collatz took: 00:00:13.781218646
----------------------------------------------------------------------------------------------------
batch_size # => 62500000
batches # => 16
Seed: 999000000 Steps: 162
collatz took: 00:00:13.329298756

Following is the result when built with latest master 879ec1247
total_seeds # =><PHONE_NUMBER>
worker_size # => 16
----------------------------------------------------------------------------------------------------
batch_size # => 1000
batches # => 1000000
Seed: 999000000 Steps: 162
collatz took: 00:00:13.540662927
----------------------------------------------------------------------------------------------------
batch_size # => 2000
batches # => 500000
Seed: 999000000 Steps: 162
collatz took: 00:00:13.590167441
----------------------------------------------------------------------------------------------------
batch_size # => 5000
batches # => 200000
Seed: 999000000 Steps: 162
collatz took: 00:00:13.690674863
----------------------------------------------------------------------------------------------------
batch_size # => 500
batches # => 2000000
Seed: 999000000 Steps: 162
collatz took: 00:00:13.750579132
----------------------------------------------------------------------------------------------------
batch_size # => 100
batches # => 10000000
Seed: 999000000 Steps: 162
collatz took: 00:00:14.017306410
----------------------------------------------------------------------------------------------------
batch_size # => 62500000
batches # => 16
Seed: 687000000 Steps: 239
collatz took: 00:00:22.480855790

The spent time from 13s -> 22s when run on latest master, almost 70% performance loss, it is reproducible.

Is this the same code you mentioned in https://forum.crystal-lang.org/t/there-is-a-way-to-optimize-this-program/6947/63?u=straight-shoota ? Please add at least references to related comments when cross-posting.

Is this the same code you mentioned in https://forum.crystal-lang.org/t/there-is-a-way-to-optimize-this-program/6947/63?u=straight-shoota ?

Yes, I mention it in several places in the forum, and i consider this is a regression issue now.

Thanks, I'm aware of your report. I didn't have the time to dig into it, yet.

Creating a log instance and using it instead of print "\r" also avoids the issue, even if writing much more often (every 100 000 seeds), the program never concurrently writes to STDOUT.

To sum up: I pinpointed the issue to multiple threads trying to write to the same fd. In this case it's stdout. Whenever the concurrency writes are dropped (using Log or a Channel+Fiber) the performance is back.
What I extracted from tracing is that the fd would block at some point, so it's added to epoll, then the evloop is notified that it became ready and the fiber continues... but the fd is left in the evloop.
Then said thread has nothing left to do but other threads continue writing to the fd, which regularly triggers ready events, and the thread enters a loop { evloop.wait; evoop.ready fd=4 }.
I don't know why it has an impact on the overall performance. It shouldn't have any impact on the other threads.


[Event Request] OnBeforeIsActiveProductionBOM @ Codeunit 99000770 "Where-Used Management"

I'd like to propose the Event OnBeforeIsActiveProductionBOM in Codeunit "Where-Used Management" with the Handled-Pattern. Please let me know if you need additional information from me.
local procedure IsActiveProductionBOM(ProductionBOMLine: Record "Production BOM Line"): Boolean
// START
var
    IsHandled: Boolean;
    Result: Boolean;
begin
    OnBeforeIsActiveProductionBOM(ProductionBOMLine, Result, IsHandled);
    if IsHandled then
        exit(Result);

// STOP
    if ProductionBOMLine."Version Code" = '' then
        exit(not IsProductionBOMClosed(ProductionBOMLine));

    exit(not IsProdBOMVersionClosed(ProductionBOMLine));
end;

// START
[IntegrationEvent(false, false)]
local procedure OnBeforeIsActiveProductionBOM(ProductionBOMLine: Record "Production BOM Line"; var Result: Boolean; var IsHandled: Boolean)
begin
end;
// STOP

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.


Add VSET and ISET

Currently we only have VOUT and IOUT for the live values.
We should also dump VSET and ISET so we can see what the set limits are.
Perhaps make the current and voltage commands dump this info if you don't do a set?


About DIML datasets

Hi, I download DIML datasets, but I dont find the "calib color 2-digits.mat" which intrinsic parameter for color camera. Do you know why? Thank you very much!

Hi, to run the code in this repository, only the indoor sample images are needed. Therefore, no intrinsic calibration parameters are required. If you have any questions about the dataset, you should contact the publishers of the dataset.


Show charts

Have a place in the new design for the charts that we have in current website. It doesn't have to be exactly same format but would like to keep the same chart. Please do share if you have better ideas.
https://theweeklychallenge.org/chart/

Hello @manwar, as an Outreachy applicant, Could you please confirm if contributions are still open? I'm excited about the opportunity to make valuable contributions to this project.

Yes, if I am not mistaken, the contribution period is still on. You are most welcome to this project. @manwar

Yes!!Thank you for the confirmation! @manwar  I'm thrilled about the opportunity to contribute to this project. However, I encountered an issue trying to join the Slack project's public chat. Could you please assist me with joining the Slack channel? Looking forward to collaborating with everyone!

@kanonkartik please share your email and I will send you the invite.

my email is:-<EMAIL_ADDRESS>thankyou @manwar

Just sent you invitation. @kanonkartik

@manwar ,  I would like to work on this issue please assign this to me .


Adding AnimatedIcon and Icon Control Pages

Description
Created new sample page for AnimatedIcon and added a single page for the other icon controls called Icon Controls (open to a different name). I wasn't sure if I could add a space to the page name so right now the page is IconControls. I also know that there is a bug in the second example for BitmapIcon with the binding but I wanted to create the PR now so everything else can be reviewed.

I'm seeing the app crash when I try to navigate to the AnimatedIcon page from What's New. Anyone else seeing the same thing, or is it just my machine?

I'm seeing the app crash when I try to navigate to the AnimatedIcon page from What's New. Anyone else seeing the same thing, or is it just my machine?

I think it's because I accidentally commented out code. I fixed it and I'm about to push all of my changes.

I'm seeing the app crash when I try to navigate to the AnimatedIcon page from What's New. Anyone else seeing the same thing, or is it just my machine?

For the record, it was also crashing on my end, something about not being able to find a file.

@YuliKl @chingucoding I updated the CS file for AnimatedIcon so hopefully it doesn't crash for you. It was working on my local machine.


Add contrib package for MSSQL

This is primarily based off of the mysqldb.py module, but uses the _mssql library

looks good!

Cool. Some thoughts:

Can you add some more docs?

Is it read/write only or both?
Perhaps a minimal example as module level documentation.

Can tests be added?

If you mostly copy-pasted some other module, did you consider generalizing/inheriiting and stuff??

Note, all these things are optional. This can be merged as-is too since it's under contrib/.

Hm, I think generalizing/inhereiting stuff is a good suggestion, but would probably require larger changes.
Generally, the code used for most of the methods is mostly identical to mysqldbpy and postgres.py.
One thought would be to add a abstract rdbms target class to rdbms.py.  Then refactor the postgres, mysql, and mssql libraries to leverage that abstract class.  I'm not sure if that's a change that y'all have discussed before/would be open to making.
I'm still thinking about how to make generalizable Task classes for the mssql library.  The postgres library has CopyToTable and Query, which aren't use cases that I'm working with, but I'm thinking about a RunProcedure task class.  I'm fairly new to luigi, so I'm still working on having the whole library cement itself in my mind.
Do you have any thoughts on the above, considering your much larger familiarity (duh) with the project?

Well. One decent thing to do in these situations when one doesn't have time to dive into the code base and refactor and stuff is to at least add some docs saying """This module is mostly a copy-paste of other_module.py. In the future one common interface could be derived.""". In addition to that I still think you at least should have an usage example in the module-level docs.

@Tarrasch - Added the comments suggested above.  This can be merged.


:space_invader: a11y fixes - fixes #135 fixes #134

a11y fixes

Please resolve these conflicts! ✨

fixed 💥


bug: @param type lacks syntax highlighting

Does this issue occur when all extensions are disabled?: Yes

VS Code Version: 1.89.1 (Universal)
OS Version: Darwin arm64 23.5.0

Steps to Reproduce:

Create a TypeScript file containing TSDoc.

Please let me know if I'm missing any info.

are you meaning @typeParam?

Yes, sorry, I meant @typeParam.


⚠ AVS (kty) has degraded performance

In d5ba975, AVS (kty) (https://avus.com.br) experienced degraded performance:

HTTP code: 200
Response time: 8926 ms

Resolved: AVS (kty) performance has improved in 591c4ae.


signer / crypto abstractions

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

68 of 80 (85.0%)  changed or added relevant lines in 3 files are covered.
10 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.09%) to 57.157%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

extensions/envelope/signer.go
32
34
94.12%

extensions/envelope/middleware.go
18
21
85.71%

extensions/envelope/crypto.go
18
25
72.0%

Files with Coverage Reduction
New Missed Lines
%

extensions/token/balance_utxo.go
10
69.61%

Totals

Change from base Build<PHONE_NUMBER>:
-0.09%

Covered Lines:
3993

Relevant Lines:
6986

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

68 of 80 (85.0%)  changed or added relevant lines in 3 files are covered.
33 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.1%) to 57.143%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

extensions/envelope/signer.go
32
34
94.12%

extensions/envelope/middleware.go
18
21
85.71%

extensions/envelope/crypto.go
18
25
72.0%

Files with Coverage Reduction
New Missed Lines
%

extensions/token/balance_utxo.go
10
69.61%

testing/mockstub.go
23
51.13%

Totals

Change from base Build<PHONE_NUMBER>:
-0.1%

Covered Lines:
3992

Relevant Lines:
6986

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

68 of 80 (85.0%)  changed or added relevant lines in 3 files are covered.
33 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.1%) to 57.143%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

extensions/envelope/signer.go
32
34
94.12%

extensions/envelope/middleware.go
18
21
85.71%

extensions/envelope/crypto.go
18
25
72.0%

Files with Coverage Reduction
New Missed Lines
%

extensions/token/balance_utxo.go
10
69.61%

testing/mockstub.go
23
51.13%

Totals

Change from base Build<PHONE_NUMBER>:
-0.1%

Covered Lines:
3992

Relevant Lines:
6986

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

71 of 80 (88.75%)  changed or added relevant lines in 3 files are covered.
33 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.07%) to 57.18%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

extensions/envelope/signer.go
32
34
94.12%

extensions/envelope/middleware.go
18
21
85.71%

extensions/envelope/crypto.go
21
25
84.0%

Files with Coverage Reduction
New Missed Lines
%

extensions/token/balance_utxo.go
10
69.61%

testing/mockstub.go
23
51.13%

Totals

Change from base Build<PHONE_NUMBER>:
-0.07%

Covered Lines:
3994

Relevant Lines:
6985

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

73 of 82 (89.02%)  changed or added relevant lines in 3 files are covered.
23 unchanged lines in 1 file lost coverage.
Overall coverage increased (+0.1%) to 57.349%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

extensions/envelope/signer.go
32
34
94.12%

extensions/envelope/middleware.go
18
21
85.71%

extensions/envelope/crypto.go
23
27
85.19%

Files with Coverage Reduction
New Missed Lines
%

testing/mockstub.go
23
51.13%

Totals

Change from base Build<PHONE_NUMBER>:
0.1%

Covered Lines:
4007

Relevant Lines:
6987

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

73 of 85 (85.88%)  changed or added relevant lines in 3 files are covered.
33 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.08%) to 57.167%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

extensions/envelope/middleware.go
18
21
85.71%

extensions/envelope/crypto.go
23
27
85.19%

extensions/envelope/signer.go
32
37
86.49%

Files with Coverage Reduction
New Missed Lines
%

extensions/token/balance_utxo.go
10
69.61%

testing/mockstub.go
23
51.13%

Totals

Change from base Build<PHONE_NUMBER>:
-0.08%

Covered Lines:
3996

Relevant Lines:
6990

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

76 of 88 (86.36%)  changed or added relevant lines in 3 files are covered.
33 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.06%) to 57.186%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

extensions/envelope/middleware.go
18
21
85.71%

extensions/envelope/crypto.go
23
27
85.19%

extensions/envelope/signer.go
35
40
87.5%

Files with Coverage Reduction
New Missed Lines
%

extensions/token/balance_utxo.go
10
69.61%

testing/mockstub.go
23
51.13%

Totals

Change from base Build<PHONE_NUMBER>:
-0.06%

Covered Lines:
3999

Relevant Lines:
6993

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

73 of 82 (89.02%)  changed or added relevant lines in 3 files are covered.
33 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.05%) to 57.192%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

extensions/envelope/signer.go
32
34
94.12%

extensions/envelope/middleware.go
18
21
85.71%

extensions/envelope/crypto.go
23
27
85.19%

Files with Coverage Reduction
New Missed Lines
%

extensions/token/balance_utxo.go
10
69.61%

testing/mockstub.go
23
51.13%

Totals

Change from base Build<PHONE_NUMBER>:
-0.05%

Covered Lines:
3996

Relevant Lines:
6987

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

61 of 82 (74.39%)  changed or added relevant lines in 4 files are covered.
33 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.2%) to 57.02%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

extensions/envelope/sign.go
23
25
92.0%

extensions/envelope/middleware.go
18
21
85.71%

extensions/envelope/crypto/ed25519.go
11
27
40.74%

Files with Coverage Reduction
New Missed Lines
%

extensions/token/balance_utxo.go
10
69.61%

testing/mockstub.go
23
51.13%

Totals

Change from base Build<PHONE_NUMBER>:
-0.2%

Covered Lines:
3984

Relevant Lines:
6987

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

61 of 82 (74.39%)  changed or added relevant lines in 4 files are covered.
33 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.2%) to 57.034%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

extensions/envelope/sign.go
23
25
92.0%

extensions/envelope/middleware.go
18
21
85.71%

extensions/envelope/crypto/ed25519.go
11
27
40.74%

Files with Coverage Reduction
New Missed Lines
%

extensions/token/balance_utxo.go
10
69.61%

testing/mockstub.go
23
51.13%

Totals

Change from base Build<PHONE_NUMBER>:
-0.2%

Covered Lines:
3985

Relevant Lines:
6987

💛  - Coveralls


Error when an Identifier is being broken up by buffer ending

I believe this is actually a continuation of #1, but since it's closed and I don't seem to have a way to reopen it, I'll open up a new issue.
Seems like there is still some problem with the buffering mechanism. When there is an identifier being broken up by buffer ending, it would be read as two separate identifiers causing error.

After investigating seems like I missed something the last time. If I comprehend correctly, EnsureBuffer(len) should return true when there is actually len chars to read (those many of chars can be ensured, like the method name), and false if no further chars are available. However at line 242:
https://github.com/CXuesong/Luaon.NET/blob/146db4e240df8505e17ae0d69a3b3e91e46a75d4/Luaon.NET/LuaTableTextReader.cs#L235-L243
After the if on line 237, a successful state can be assured (there is more char to read). So looks like it should return true instead. I tested locally with the file that I need to parse and seems like it's working.

Nice catch! Fixed and released v0.2.4.


prefix query

Hi,
Great work on Tire... I'll probably be going through and attempting to add a few more query types and maybe 1.9 support for tests, etc.  Hope this will be ok with you!
Anyway - here's my first one to support the prefix query type.
Thanks,
Topper

Hi, thanks!

attempting to add a few more query types

yeah, feel fee to do so.

and maybe 1.9 support for tests

not sure what you mean by that?

here's my first one to support the prefix query type.

I'd like to have query types covered by integration tests as well, is it possible to add some?
Best,
Karel

added a integration test


refactor: Use dxos_echo_schema.Document inside Kai StackFrame

Current dependencies on/for this PR:

main

PR #2646   👈

This comment was auto-generated by Graphite.


Create Profile Box

Profile box for user

Please accept for hacktoberfest

@alphaninja27 please attach a screenshot of the changes you made.

This is a Profile Box structure, where users name picture and brief description appears in an aesthetic format, the code is the structure for the same


Add method returning client provided file path

UploadedFile class has a property containing temporary path of the
uploaded file, which is inaccessible outside of the class. But there is
a need of accessing this property outside of the class, when for
example you must upload the uploaded file directly to the cloud(e.g.
Amazon S3 bucket) without saving it to your local file system. So a
getter method is created returning the value of this property.

Coverage remained the same at 100.0% when pulling 346854bdc5a54d3bd4701ea8fbebdbe66063c651 on hhovakimyan:feature/add-method-returning-file-path-in-UploadedFile-class into 5b1913319473423cedce8c51b4bbb455dfcb6a4a on slimphp:master.

@l0gicgate can you please merge this PR as soon as possible?

@hhovakimyan this is not part of the PSR-7 spec, that's why it has not been implemented. Unfortunately I don't think this is necessary, it seems like this is only for the edge case in your application.
@akrabat what are your thoughts on this?

@l0gicgate first I don't think the case in my application is an edge case. I think many people would like to send the uploaded file directly to cloud(AWS S3 bucket, Google Drive, Dropbox, etc) without copying it to local file system and then deleting the local copy after upload succeeds. Second, if yo want to strictly follow the PSR-7 spec, there is also the option of changing the access modifier of $file property from protected to public. In Slim 3 the $file property of UploadedFile class was a public property and I used it for getting the value of uploaded file's tmp path. That's why I have a backward compatibility issue after migrating from Slim 3 to Slim 4.

@l0gicgate thanks a lot for merging the pull request. Sorry for the way I have asked about merging this PR. Next time I will use suggestive language in such cases. Also can you tell me please when the 1.2 milestone is planned to be released?

@hhovakimyan just released for you:
https://github.com/slimphp/Slim-Psr7/releases/tag/1.2.0

Thanks @hhovakimyan.
By the way I think this should be part of a PSR-7 spec, for the reasons you outlined.


Deprecate release_notes.md requirement

@MehdiK, @mexx  as discussed in twitter

:+1:  Thanks for the change.
Dear contributor,
Please do NOT add your PR to the release note file from now on.
Yours truly,
The Humanizer Team


Not Lazy Loading Dynamic

The library does not lazy load dynamically added images.
Does one have to specifically apply the script to the newly added images?
Thanks

@hn1 Have you tried using this plugin?

What plugin!?

If you add images dynamically it should work with the core script out of the box. In case you dynamically want to change src attributes you need the attr change plugin.
If you still experience problems provide a test case.


How can i log out for both FB and G+

You can find a logout method in UserSessionManager. You can use it to remove the current user session. Thinking of updating the documentation and wiki along with the next version. So meanwhile take look into the above class to find out more.


refactor: upgraded code to es6 standard

Changes:
Improved eslint configuration.
Upgraded code to es6 standard.
Optimized use of arrays.
Since you are already using ES6 features (Uint8Array as example), I allowed myself to adapt your code to this standard.
All var were replaced by let or const. So we will give more optimization opportunities for JS engine.
The same goes for using arrays. It is not recommended to create arrays through new Array(N).  Which affects performance.
https://v8.dev/blog/elements-kinds
Since you are already using Uint8Array, I replaced regular arrays with typed ones (in some places). Which are created with a fixed size and free from the disadvantages of regular arrays.
P.S. If something does not suit you in my changes, then let me know and I will fix it =)

@ctavan if you want me to make a benchmark, then I will do it. But a little later.
What test tools do you prefer? or is it not that important?

@ctavan if you want me to make a benchmark, then I will do it. But a little later.
What test tools do you prefer? or is it not that important?

Great! I haven't done performance testing of JavaScript libraries recently so I would have to investigate what the state-of-the art is.
The only thing that I would consider important is that performance can be tested both, in Node.js as well as in browsers.
The performance tests should probably also be committed to this repository.

exploring ways of benchmarking the library

For the record, I used to track performance in this module  (using ad-hoc perf tests).  I removed that code because perf simply stopped being interesting.   It's been a while but last I checked I think the #'s were on the order of 100K's or even 1M's of ids/second (for v1 and v4).
I eventually concluded that the overhead of UUID creation almost always paled in comparison to whatever else the user was doing with that UUID (e.g. creating a DB entry or creating/validating model data).
That said, having a performance test as part of our CI tests to catch order-of-magnitude regressions would be good, but I don't think we need it as part of this PR.  Let's create a separate issue for that.

@broofa I totally agree that performance is currently most likely not a problem of the uuid library. All I want is to catch, as you write, "order-of-magnitude regressions".
I've added a little script that might already do the trick: #430
Would be curious to see it run before and after this PR. @awwit wanna give it a try?

@ctavan I also already managed to prepare the code for performance testing (also using benchmark).
Almost the same code turned out as yours. I have to try it.
(In my version, I output the results to a file ${git-branch}-${node--version}.json. So you can save test results and switch to other branches.)
Let's try your benchmark. You need to merge it into a master. Then I can merge it into my branch and see the difference in performance.

@awwit I've merged the benchmark script to master. Could you rebase this PR so we can see the results?
Thanks!

My one concern is that embracing ES6 is going to encourage the use of patterns and features that may break IE11, where there's limited ES6 support. E.g. see my comment below about using a default value for the offset argument. Such bugs should get caught by our BrowserStack tests though (right, @ctavan?)

Regarding JS compatibility: Since we use babel to transpile the source into a form to support all platforms that we want to support we should not need to worry too much.
Also our tests should catch regressions for all currently supported platforms 🤞 ...
The only thing we should be careful about is that the more we let babel transpile code, the higher the risk that we get

increased bundlesize due to poor transformations of babel (we have bundlewatch to catch that!)
performance regressions (we have the benchmark script now to catch that!)

@ctavan done, I merged your tests to my branch.
Now we can look at the performance.

Node 12
Master:
Starting. Tests take ~1 minute to run ...
uuidv1() x 1,319,851 ops/sec ±1.76% (86 runs sampled)
uuidv1() fill existing array x 4,732,686 ops/sec ±1.83% (90 runs sampled)
uuidv4() x 294,904 ops/sec ±2.72% (82 runs sampled)
uuidv4() fill existing array x 357,955 ops/sec ±3.05% (82 runs sampled)
uuidv3() x 106,752 ops/sec ±3.61% (79 runs sampled)
uuidv5() x 113,064 ops/sec ±1.56% (83 runs sampled)
Fastest is uuidv1() fill existing array

This branch:
Starting. Tests take ~1 minute to run ...
uuidv1() x 1,519,697 ops/sec ±2.50% (87 runs sampled)
uuidv1() fill existing array x 8,119,318 ops/sec ±2.89% (92 runs sampled)
uuidv4() x 303,734 ops/sec ±2.53% (86 runs sampled)
uuidv4() fill existing array x 362,363 ops/sec ±4.08% (85 runs sampled)
uuidv3() x 114,053 ops/sec ±1.37% (82 runs sampled)
uuidv5() x 108,882 ops/sec ±2.21% (80 runs sampled)
Fastest is uuidv1() fill existing array

uuidv1() performance has improved, especially in the "fill existing array" case 😲 ! Would be curious to learn what caused this improvement.
The rest seems to perform similar than before (within error margins) 👏

Thanks for your effort! ... this is a valuable contribution and will help us modernize the code base. ... I think in general this already looks really good!

Agreed!

uuidv1() performance has improved, especially in the "fill existing array" case 😲 ! Would be curious to learn what caused this improvement.

Me too.  I know switching from var to const/let is generally considered a good thing (perf-wise).  As is using typed arrays, but I'd be curious to hear what someone familiar with how JS engines work would have to say about this.

@ctavan

uuidv1() performance has improved, especially in the "fill existing array" case  ! Would be curious to learn what caused this improvement.

I checked, this is due to the fact that we replaced new Date().getTime() to Date.now().

@ctavan Bundlewatch test failed =/

We will have to increase the size to get more optimizations (especially parsing uuid bytes).
If the size is so critical, then I can remove typed arrays in failed bundles.

I think we are starting to change too much stuff in this PR, this no longer feels controllable.
Could you please revert your latest code optimizations (let's discuss them in a separate pull request)?
I would then try to get this merged quickly so that we can go through further code optimizations in a more step-by-step manner and therefore controlling better for possible performance optimizations.
So for the time being I'd prefer if this PR would contain only:

code style changes (eslint config)
let/const changes
typed array changes

Let's move all other code changes into a separate pull request.

Oh, and to get this in it would be great if you could Allow changes to a pull request branch created from a fork. This would help me get this merged and keep all references to your contributions.

Benchmark master:
Starting. Tests take ~1 minute to run ...
uuidv1() x 1,377,681 ops/sec ±1.47% (91 runs sampled)
uuidv1() fill existing array x 4,881,829 ops/sec ±2.99% (89 runs sampled)
uuidv4() x 306,412 ops/sec ±2.56% (85 runs sampled)
uuidv4() fill existing array x 362,883 ops/sec ±3.90% (85 runs sampled)
uuidv3() x 111,873 ops/sec ±2.63% (81 runs sampled)
uuidv5() x 110,720 ops/sec ±2.73% (82 runs sampled)
Fastest is uuidv1() fill existing array

Benchmark this branch:
Starting. Tests take ~1 minute to run ...
uuidv1() x 1,555,686 ops/sec ±2.63% (90 runs sampled)
uuidv1() fill existing array x 7,804,665 ops/sec ±1.92% (87 runs sampled)
uuidv4() x 311,104 ops/sec ±2.22% (82 runs sampled)
uuidv4() fill existing array x 364,342 ops/sec ±4.73% (86 runs sampled)
uuidv3() x 117,100 ops/sec ±3.78% (82 runs sampled)
uuidv5() x 115,292 ops/sec ±1.76% (83 runs sampled)
Fastest is uuidv1() fill existing array

@awwit I just realized that I already hat push access to your branch. So I did the requested cleanups and merged your changes 🎉
I copied the advanced optimizations to https://github.com/uuidjs/uuid/tree/awwit-refactorings feel free to pick them up from there and create another pull request if you want.
I'm happy to merge those changes as well once I understand their benefit and performance implications.

@ctavan Fine! Then I will continue to do in another brunch. I will remove the code that greatly inflates the bundle. And I’ll do something else.


docker-slim: 1.40.3 -> 1.40.4

Diff: https://github.com/slimtoolkit/slim/compare/1.40.3...1.40.4
Changelog: https://github.com/slimtoolkit/slim/raw/1.40.4/CHANGELOG.md
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[x] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.11 Release Notes (or backporting 23.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 251606 run on x86_64-linux 1

1 package built:

docker-slim


Android backend as android library

This pr makes an android library of the android backend. This has the advantage, that if someone increases their libgdx version in their build.gradle file and the minsdk is lower than the libgdx one, it throws an error. The next advantage is a better IDE support for the android backend.
The renaming of the modules fixes also this problems:

Is an "android library" an AAR file, as opposed to the current JAR? Does Sonatype Maven Central distribute AARs (it looks like this does handle Travis for nightly builds)? Do Eclipse, Netbeans, and command-line Maven have the capability to build libGDX with this PR, or does this restrict builds to IDEA/AS/command-line Gradle? It's clear that Gradle wasn't used for recent nightlies because this PR changed the version from 1.9.9-SNAPSHOT to the current one, unless the version was changed elsewhere.

The release can be done with maven like before. The goal is to get to the point where we have only gradle to publish. Sonatype maven central supports android libraries. Here is for example a tutorial how to do it. If an IDE does not support the "new" (regular) build system for building android libs, you can use the old way. I did not remove this option

Hey @Tom-Ski, some guys told me on discord told me that this PR got to much changes. It fixes the error from the screenshot above, makes it possible to deploy aar's (Old method is still possible) and migrates to androidx. Are you agree with them? If so, I would make a PR which fixes only the unique path problem with the android gradle plugin. This means still a huge PR, because I have to rename some modules. Will you accept the PR then? I don't want put effort in it if it doesn't get pulled (like some of my other PR's)

Mainly I can't reproduce the problems you are having here, I don't have issues importing and building from IDE, other than a resource problem with IDEA marking directories as non-source, but resource.
I like the goal, of only having Gradle to publish, but we are pretty far away from this with the native builds. And last time I checked, aar had pretty poor support in other IDEs.

about folder renaming, i can confirm there is an issue:

i'm using eclipse with the old import method (based on git tracked eclipse files) without issues.
i tried the other method (via gradle) using a fresh Eclipse 2019.12 (latest), a fresh libgdx repo and follow official setup instructions. I can confirm i have a similar issue :

both box2d and controllers projects are correctly imported but are weirdly renamed (eg. gdx-controllers-gdx-controllers).
however, dependant projects (eg. gdx-test-lwjgl) reference it incorrectly (eg. gdx-controllers) and reported as missing dependencies.

note that git tracked eclipse files is an issue when importing via gradle: all .project and .classpath files are modified so it's annoying to work with. By fixing folder names, i think we could also remove all eclipse files from repo and remove the old eclipse import method.

I also see the issue affecting the 4 modules described in the PR on IDEA 2019.3.3 but the error plays a bit of seek and hide and sometimes doesn't appear even if it's there and pops up at times.
Importing a totally clean checked out libGDX project displays the project. Also, it ususally shows running a "Reimport All Gradle Projects" from IDEA Gradle menu.

@Tom-Ski As you can see, more users have the same problem. Sadly, I can't really tell you how to reproduce the error because it appears directly at the start of IntelliJ, like obigu said. Maybe you have to do something with android? Because it's problem with the android gradle plugin. I had a similar error on travis, while making the android backend an android library. Maybe that helps? https://travis-ci.com/github/SimonIT/libgdx/jobs/256907200

Yes I've spoken to @obigu about this, but with unrelated fixes he is able to import ok? (just some other problems to fix also)

Yes, somehow it runs.
In my opinion, a functional build system is a must have for a library/framework and the current gradle setup is only more or less working. Like Nathan said, libgdx is counting on PR contributions because "it's in maintenance mode". I saw on discord several times persons who ask how to fix these errors, because they think it's their setup which causes the problem. So it scares developers away from contributing which not very great for libgdx.
Another point is, as you said, to have full deployable gradle setup. If we want the android backend as normal android library (I think it's the way to go), we must do this change, because the build will fail as you can see in the previous linked travis build.

Yes, at the moment there are 2 separate issues affecting IDE integration, one is related to resources (no issue number created for it) and makes it impossible to work with IDEA and the other is the one this PR fixes which is not that critical but we definitely also want to fix.
IMO this PR goes on the right direction but I'd come back to discuss about the option of splitting it on 3 different PRs when we get the resources issue fixed as it will also probably require a large refactor.


Templates for Issues

This pull request is intended to resolve issue https://github.com/cncastillo/KomaMRI.jl/issues/228

Change feature request to .yaml

Change feature request to .yaml

Addressed in https://github.com/cncastillo/KomaMRI.jl/pull/254/commits/d2346699665f93e61aeb7186010542713382515a

Put in an issue al the warnings that we are getting for the docs.

Put in an issue al the warnings that we are getting for the docs.

Addressed in https://github.com/cncastillo/KomaMRI.jl/issues/255


Update HttpContext.TraceIdenfier with Telemetry.Name

Update HttpContext.TraceIdentifier with Telemetry.Name generated in MvcDiagnosticsListener.
The motivation of the change is described in #321

Hi @northtyphoon, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Bin Du). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla.microsoft.com.

TTYL, MSBOT;

I'm closing this PR as I do not agree with approach:

Our SDK is trying to be "least" invasive. Attaching the name to trace identifier will not be expected by other telemetry solutions.
We already misuse diagnostics source for http headers injection. We discussed it explicitly and agreed that it is not invasive.
Request name can be changed during the action execution. So this name may not be a final one.
There are other properties besides the name that can be used for correlation between SP and AI. So the approach of receiving of entire RequestTelemetry is better

I'll duplicate this response in the original issue.


[ACM-6981] Added olmv1alpha1 to scheme

Adding olmv1alpha1 to scheme to register the kind for ClusterServiceVersion. Since this was missing, when DISABLE_CLIENT_CACHE is enabled, this will cause the multicluster-engine-operator pod to fail.
1.6934115950189047e+09 ERROR setup unable to start manager {"error": "no kind is registered for the type v1alpha1.ClusterServiceVersion in scheme \"pkg/runtime/scheme.go:100\""}

/lgtm


InfluxDB-python version 5.3.0 chunk=True

InfluxDB-python version: 5.3.0
Python version: 3.7.4
Operating system version: macOS 10.14.5

msgpack.exceptions.ExtraData: unpack(b) received extra data
Traceback (most recent call last):
  File "/Users/dong/Desktop/mosaic-research/analysis/analysis.py", line 17, in <module>
    public_book = mosaic_client.public_book(exchange=exchange, instrument=instrument, ts_start=ts, ts_end=ts+save_interval, depth=1)
  File "/Users/dong/Desktop/mosaic-research/py_mosaic_client/py_mosaic_client/mosaic_client.py", line 74, in public_book
    result = self.client.query(f'SELECT * FROM "l2_book-{exchange}" WHERE time > {ts_start} AND time <= {ts_end}', chunked=True, chunk_size=10000)
  File "/Users/dong/opt/anaconda3/lib/python3.7/site-packages/influxdb/client.py", line 518, in query
    expected_response_code=expected_response_code
  File "/Users/dong/opt/anaconda3/lib/python3.7/site-packages/influxdb/client.py", line 352, in request
    raw=False)
  File "msgpack/_unpacker.pyx", line 209, in msgpack._cmsgpack.unpackb
msgpack.exceptions.ExtraData: unpack(b) received extra data.

https://github.com/influxdata/influxdb-python/commit/c903d73efcf49b4e340490072d777d8f34ac8e1c
I think it may be related to this PR

Thanks for reporting this @xiandong79, I'll investigate ASAP. I should have added a test to the dataframe_client for this.

I can take a look too if that helps, I haven't come across that issue though.

the version 5.2.3. works well

I'm having the same issue querying from both Influx 1.7.10 and 1.7.7
Interestingly with Influx 1.0.2 the bug is not present.

There are a lot of differences between 5.2.3 and 5.3.0, which is why we stepped a minor release instead of point-release.
@hrbonz if you want to take a look that would be AWESOME!

I am getting a different error, but seemingly from a similar place.

influxdb/client.py in request(self, url, method, params, data, stream, expected_response_code, headers)
    350                 packed=response.content,
    351                 ext_hook=_msgpack_parse_hook,
--> 352                 raw=False)
    353         else:
    354             response._msgpack = None

msgpack/_unpacker.pyx in msgpack._cmsgpack.unpackb()
`UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 8: invalid continuation byte`

Similar with SHOW DIAGNOSTICS query

InfluxDB 1.7.6
InfluxDB-python version: 5.3.0
Python version: 3.6.9
Operating system version: Ubuntu 18.04

InfluxDB-python version: 5.3.0
Python version: 3.7.4
Operating system version: macOS 10.14.5
python3
Python 3.6.9 (default, Apr 18 2020, 01:56:04)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from influxdb import client
>>> influxdb_client = client.InfluxDBClient("<IP_ADDRESS>", "8086")
>>> influxdb_client.ping()
'1.7.6'
>>> influxdb_client.query('SHOW DIAGNOSTICS')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/vagrant/.local/lib/python3.6/site-packages/influxdb/client.py", line 518, in query
    expected_response_code=expected_response_code
  File "/home/vagrant/.local/lib/python3.6/site-packages/influxdb/client.py", line 352, in request
    raw=False)
  File "msgpack/_unpacker.pyx", line 213, in msgpack._cmsgpack.unpackb
ValueError: Unpack failed: incomplete input

With InfluxDB-python version: 5.2.3 the bug is not present.

I can confirm query with chunk=True does not work on 5.3.0.

Hello Team,
Any workaround for this issue?

Hello Team,
Any workaround for this issue?

Sure use <5.3.0

having the same issue - any progress?

Hi,
Having same issue. Any solution ?
Debian GNU/Linux 9.4 (stretch)
python 2.7.13
Influx       1.8.3
Influxdb   5.3.1
msgpack 1.0.2
msgpack.exceptions.ExtraData: unpack(b) received extra data.
Traceback (most recent call last):
File "/code/apps/FuelChangeoverPlot.py", line 179, in exportFromDb
data = data_fetcher.fetch_fuel_change_over_plot(start_time=rangeStart, end_time=rangeEnd)
File "/code/db_interface/data_fetcher.py", line 35, in fetch_fuel_change_over_plot
df_dict = db_connector.query_for_single_measurement_range(
File "/code/db_interface/db_connector.py", line 80, in query_for_single_measurement_range
df_dict = client.query(
File "/usr/local/lib/python3.9/site-packages/influxdb/_dataframe_client.py", line 199, in query
results = super(DataFrameClient, self).query(query, **query_args)
File "/usr/local/lib/python3.9/site-packages/influxdb/client.py", line 521, in query
response = self.request(
File "/usr/local/lib/python3.9/site-packages/influxdb/client.py", line 358, in request
response._msgpack = msgpack.unpackb(
File "msgpack/_unpacker.pyx", line 202, in msgpack._cmsgpack.unpackb

There are actually two issues here:

Unpack issue when using msgpack. I did not debug this further, but here's a workaround that works for me: use json instead of msgpack. This can be forced using:
InfluxDBClient('ip', 8086, '', '', 'database', headers={'Accept': 'application/json'}, gzip=True)

Even when using the above, DataFrameClient does not work.

There are actually two issues here:

Unpack issue when using msgpack. I did not debug this further, but here's a workaround that works for me: use json instead of msgpack. This can be forced using:
InfluxDBClient('ip', 8086, '', '', 'database', headers={'Accept': 'application/json'}, gzip=True)

Even when using the above, DataFrameClient does not work.

Debian GNU/Linux bullseye/sid
Python 3.9.2
influxdb-python master branch
Influxdb 1.8.5 and 1.7.3
msgpack 1.0.2
Run my test scripts with export MSGPACK_PUREPYTHON=1 to use python implementation of msgpackrather than the C, easier for debugging.
Analysis
I've looked into this issue today, it looks to me like a combination of two problems:

I forgot to add the stream/chunk changes to DataFrameClient because I didn't even realize it existed, I'll submit a PR with the proper changes for it.

If I run a 'SHOW DIAGNOSTIC' with headers set to accept JSON, I get the following:

{
  "results": [
    {
      "statement_id": 0,
      "series": [
        {
          "name": "build",
          "columns": [
            "Branch",
            "Build Time",
            "Commit",
            "Version"
          ],
          "values": [
            [
              "1.7",
              "",
              "ff383cdc0420217e3460dabe17db54f8557d95b6",
              "1.7.8"
            ]
          ]
        },
        {
          "name": "config",
          "columns": [
            "bind-address",
            "reporting-disabled"
          ],
          "values": [
            [
              "<IP_ADDRESS>:8098",
              true
            ]
          ]
        },
        {
          "name": "config-coordinator",
          "columns": [
            "log-queries-after",
            "max-concurrent-queries",
            "max-select-buckets",
            "max-select-point",
            "max-select-series",
            "query-timeout",
            "write-timeout"
          ],
          "values": [
            [
              "0s",
              0,
              0,
              0,
              0,
              "0s",
              "10s"
            ]
          ]
        },
        {
          "name": "config-cqs",
          "columns": [
            "enabled",
            "query-stats-enabled",
            "run-interval"
          ],
          "values": [
            [
              true,
              false,
              "1s"
            ]
          ]
        },
        {
          "name": "config-data",
          "columns": [
            "cache-max-memory-size",
            "cache-snapshot-memory-size",
            "cache-snapshot-write-cold-duration",
            "compact-full-write-cold-duration",
            "dir",
            "max-concurrent-compactions",
            "max-index-log-file-size",
            "max-series-per-database",
            "max-values-per-tag",
            "series-id-set-cache-size",
            "wal-dir",
            "wal-fsync-delay"
          ],
          "values": [
            [
<PHONE_NUMBER>,
              26214400,
              "10m0s",
              "4h0m0s",
              "/var/lib/influxdb/data",
              0,
              1048576,
              1000000,
              100000,
              100,
              "/var/lib/influxdb/wal",
              "0s"
            ]
          ]
        },
        {
          "name": "config-httpd",
          "columns": [
            "access-log-path",
            "bind-address",
            "enabled",
            "https-enabled",
            "max-connection-limit",
            "max-row-limit"
          ],
          "values": [
            [
              "",
              ":8096",
              true,
              false,
              0,
              0
            ]
          ]
        },
        {
          "name": "config-meta",
          "columns": [
            "dir"
          ],
          "values": [
            [
              "/var/lib/influxdb/meta"
            ]
          ]
        },
        {
          "name": "config-monitor",
          "columns": [
            "store-database",
            "store-enabled",
            "store-interval"
          ],
          "values": [
            [
              "_internal",
              true,
              "10s"
            ]
          ]
        },
        {
          "name": "config-precreator",
          "columns": [
            "advance-period",
            "check-interval",
            "enabled"
          ],
          "values": [
            [
              "30m0s",
              "10m0s",
              true
            ]
          ]
        },
        {
          "name": "config-retention",
          "columns": [
            "check-interval",
            "enabled"
          ],
          "values": [
            [
              "30m0s",
              true
            ]
          ]
        },
        {
          "name": "config-subscriber",
          "columns": [
            "enabled",
            "http-timeout",
            "write-buffer-size",
            "write-concurrency"
          ],
          "values": [
            [
              true,
              "30s",
              1000,
              40
            ]
          ]
        },
        {
          "name": "network",
          "columns": [
            "hostname"
          ],
          "values": [
            [
              "db01"
            ]
          ]
        },
        {
          "name": "runtime",
          "columns": [
            "GOARCH",
            "GOMAXPROCS",
            "GOOS",
            "version"
          ],
          "values": [
            [
              "amd64",
              2,
              "linux",
              "go1.11"
            ]
          ]
        },
        {
          "name": "system",
          "columns": [
            "PID",
            "currentTime",
            "started",
            "uptime"
          ],
          "values": [
            [
              10884,
              "2021-04-26T09:59:01.187859258Z",
              "2021-04-26T08:10:39.214602676Z",
              "1h48m21.973256582s"
            ]
          ]
        }
      ]
    }
  ]
}

When running without any headers, we get msgpack back with the following:
b'\x81\xa7results\x91\x82\xacstatement_id\x00\xa6series\x9e\x83\xa4name\xa5build\xa7columns\x94\xa6Branch\xaaBuild Time\xa6Commit\xa7Version\xa6values\x91\x94\xa31.7\xa0\xd9(ff383cdc0420217e3460dabe17db54f8557d95b6\xa51.7.8\x83\xa4name\xa6config\xa7columns\x92\xacbind-address\xb2reporting-disabled\xa6values\x91\x92\xae<IP_ADDRESS>:8098\xc3\x83\xa4name\xb2config-coordinator\xa7columns\x97\xb1log-queries-after\xb6max-concurrent-queries\xb2max-select-buckets\xb0max-select-point\xb1max-select-series\xadquery-timeout\xadwrite-timeout\xa6values\x91\x97\x00\x00\x00\x00\x83\xa4name\xaaconfig-cqs\xa7columns\x93\xa7enabled\xb3query-stats-enabled\xacrun-interval\xa6values\x91\x93\xc3\xc2\x83\xa4name\xabconfig-data\xa7columns\x9c\xb5cache-max-memory-size\xbacache-snapshot-memory-size\xd9"cache-snapshot-write-cold-duration\xd9 compact-full-write-cold-duration\xa3dir\xbamax-concurrent-compactions\xb7max-index-log-file-size\xb7max-series-per-database\xb2max-values-per-tag\xb8series-id-set-cache-size\xa7wal-dir\xafwal-fsync-delay\xa6values\x91\x9c\xb6/var/lib/influxdb/data\x00\xd2\x00\x0fB@\xd2\x00\x01\x86\xa0d\xb5/var/lib/influxdb/wal\x83\xa4name\xacconfig-httpd\xa7columns\x96\xafaccess-log-path\xacbind-address\xa7enabled\xadhttps-enabled\xb4max-connection-limit\xadmax-row-limit\xa6values\x91\x96\xa0\xa5:8096\xc3\xc2\x00\x00\x83\xa4name\xabconfig-meta\xa7columns\x91\xa3dir\xa6values\x91\x91\xb6/var/lib/influxdb/meta\x83\xa4name\xaeconfig-monitor\xa7columns\x93\xaestore-database\xadstore-enabled\xaestore-interval\xa6values\x91\x93\xa9_internal\xc3\x83\xa4name\xb1config-precreator\xa7columns\x93\xaeadvance-period\xaecheck-interval\xa7enabled\xa6values\x91\x93\xc3\x83\xa4name\xb0config-retention\xa7columns\x92\xaecheck-interval\xa7enabled\xa6values\x91\x92\xc3\x83\xa4name\xb1config-subscriber\xa7columns\x94\xa7enabled\xachttp-timeout\xb1write-buffer-size\xb1write-concurrency\xa6values\x91\x94\xc3\xd1\x03\xe8(\x83\xa4name\xa7network\xa7columns\x91\xa8hostname\xa6values\x91\x91\xa4db01\x83\xa4name\xa7runtime\xa7columns\x94\xa6GOARCH\xaaGOMAXPROCS\xa4GOOS\xa7version\xa6values\x91\x94\xa5amd64\x02\xa5linux\xa6go1.11\x83\xa4name\xa6system\xa7columns\x94\xa3PID\xabcurrentTime\xa7started\xa6uptime\xa6values\x91\x94\xd1*\x84\xc7\x0c\x05\x00\x00\x00\x00`\x86\x8e\xe5\x12J\xde\xab\xc7\x0c\x05\x00\x00\x00\x00`\x86u\x7f\x0c\xca\x93\xb4\xb21h48m22.092293879s'

Both should be representing the same data but the config-coordinator structure doesn't include all the values:
x83\xa4name\xb2config-coordinator\xa7columns\x97\xb1log-queries-after\xb6max-concurrent-queries\xb2max-select-buckets\xb0max-select-point\xb1max-select-series\xadquery-timeout\xadwrite-timeout\xa6values\x91\x97\x00\x00\x00\x00\x83\xa4name\xaaconfig-cqs
We can see here by the end of the string, we have \x97 that defines a 7 entries 'fixarray' but we're getting only three zeroes (\x00) before seeing an \x83 that should start the next data structure ('config-cqs').
For this reason, I believe the bug actually exists server side. That might be a similar issue generated when doing a regular query, I couldn't figure it out. I'm also not extra comfortable with go so couldn't really find where this is implemented in the server.
This behavior appeared soon after my commit because 7fb5e946062dd36a84801e4a03012a3c032a70db changed the default headers to request msgpack instead of the default JSON.
Summary
I should push a PR to implement the fixed chunked behavior in DataFrameClient.
I suspect there is a bug with the msgpack implementation server side but can't help with this. I think someone with better go knowledge should dig on that one.

@sebito91

Tried to do the request directly on the line with curl and still got a messed up msgpack answer with the same issue:
$ curl -G 'http://localhost:8096/query' --data-urlencode q='SHOW DIAGNOSTICS'  --header "Accept: application/x-msgpack" --header "Content-Type: application/json" -u root --output response.txt

@hrbonz
May be i am asking a silly question here.
Above fix is part of current released library or future release.
If future when it is expected to release?
As i tested today i still get below issue.
msgpack.exceptions.ExtraData: unpack(b) received extra data.

For any future readers, this query works and doesnt throw msgpack.exceptions.extradata: unpack(b) received extra data. Please note that I am not using the DataFrameClient.
This query in my case fetches around 6.67mil points and takes 403.592 seconds.
client = InfluxDBClient(host=host, port=port, username=user, password=password, database=dbname)
start_time = time.monotonic()
res = pd.DataFrame(client.query("select * from X where time > now() - 30m", chunked=True).get_points())
end_time = time.monotonic()
with outlock:
     print("Result from {} took {}".format(host,end_time-start_time))
     print(res)

Versions used
python --version = Python 3.7.8
influxdb.__version__ = 5.2.3

Still get ExtraData: unpack(b) received extra data., but after trying @KirannBhavaraju suggestion it worked!
Only thing I did was to remove thechunk_size=xxxx argument.
client = InfluxDBClient(blah blah)
client.query(q, chunked=True)

python = "^3.8"
influxdb = "5.3.1"


[Feature] Dependency injection without resorting to global state

Hi there! I'm starting the FastAPI web server via Uvicorn after completing various initialization steps (reading command line arguments, reading a configuration file, setting up shared memory etc.). Now I would like to access said dependencies (command line arguments, config, shared memory, …) in my endpoints or, more specifically, pass them as argument to my endpoints. Unfortunately, the current way of injecting dependencies into endpoints via Depends basically amounts to using global state, as the dependency has to be declared in the endpoint's function signature as a Callable and so has to be available already when Python runs the semantic analysis on the code.
At the same time, app.include_router(my_router, additional_dependencies=[ ]) does not solve this issue, either, as AFAIK there is no way to access these dependencies inside my endpoints.
What's the best practice to solve this issue? Ideally I would like to do something like
from dataclasses import dataclass

from fastapi import FastAPI, APIRouter

@dataclass
class MyConfig:
    foo: str = ""

router = APIRouter()

@router.post("/")
def my_endpoint(my_config: MyConfig = DependsOnType(MyConfig)):
    ...

def create_app(config: MyConfig) -> FastAPI:
    app = FastAPI()
    app.register_dependency(type=MyConfig, value=config)
    app.include_router(router, prefix="/myrouter")
    return app

Then I could do app = create_app(MyConfig(foo="bar")) to create different versions of the app, depending on the configuration and other dependencies.
EDIT: While there is a way to achieve this (see Kludex's comment below), it has various deficiencies (see the discussion below) and I would like to propose the above functionality involving DependsOnType and register_dependency() as a new feature.

The last two sections can help you: https://www.starlette.io/applications/#storing-state-on-the-app-instance
Store in app.state, and get from the request.app.

Ohh, thank you so much!
I think my suggestion from above involving DependsOnType() would still be desirable, though, because, unless I'm mistaken, using app.state breaks the type hints.

In essence you want a singleton, which isn't currently supported by the dependency injection system in fastapi.
I wrote a quick implementation for my own system though, you're welcome to use it: https://github.com/Mause/media/blob/master/rarbg_local/singleton.py#L33

I don't want a singleton, at least not in the strict sense. I just want to pass an instance of an arbitrary class (whether it's a singleton class or not) to all the endpoints in my FastAPI app.
As for your implementation, could you give an example on how to use your code?

Perhaps you could explain why what you want isn't a singleton?

@singleton
def get_settings():
    return Settings()  # we only want one instance of this to exist

@app.get('/')
def index(settings = Depends(get_settings)):
     ....

Ok, looks like we meant different things here. A singleton to me is a class of which only one instance can exist at a given time. Of course there are many classes of which there only is one single instance at a given time but these are not necessarily singletons.

Yep, different meanings for different implementations.
Maybe you could expand what you mean by runtime profile?
I didn't use @cache as that would cache it globally within the python interpreter - this implementation essentially caches it per FastAPI instance, so if you create a new FastAPI instance per test, your tests won't leak state (for example).

I didn't use @cache as that would cache it globally within the python interpreter - this implementation essentially caches it per FastAPI instance, so if you create a new FastAPI instance per test, your tests won't leak state (for example).

Oh, I see. That makes sense! :)

Maybe you could expand what you mean by runtime profile?

In my particular case, the FastAPI process gets started by a parent process which starts the whole application stack. This parent process loads a configuration file, among others, and converts it into an instance of a certain Configuration class, which it then passes on to the web server via the args argument in multiprocessing.Process() like so:
from dataclasses import dataclass
from multiprocessing import Process

from fastapi import FastAPI
import uvicorn

@dataclass
class Configuration:
    host: str
    port: int
    # …

# This function is invoked by the parent process
def start_webserver_process(config: Configuration):
    process = Process(
        target=webserver_process,
        name="Webserver process",
        args=(config, ),
    )
    process.start()

def webserver_process(config: Configuration):
    app = FastAPI()
    app.include_router(...)

    # Inject config object since some endpoints need it
    app.state.config = config

    uvicorn.run(
        app, host=config.host, port=config.port,
    )

As you can see, I cannot define a get_settings() singleton at "compilation" time as the configuration won't be known until runtime. I also don't want to set the config as module-global state in my web server process, as the state would leak during automated tests.

You could follow a similar pattern as my singleton function, like so:

def get_configuration():
    raise NotImplementedError("Configuration not injected")

def webserver_process(config: Configuration):
    ...

    app.dependency_overrides[get_configuration] = lambda: config

    ....

This let's you keep type safety as well

Yeah, I ended up doing something very similar:
from typing import Type, TypeVar

from fastapi import FastAPI

T = TypeVar("T")

def inject_dependency(app: FastAPI, type_: Type[T], value: T):
    if not hasattr(app.state, "dependencies"):
        app.state.dependencies = {}

    if not isinstance(value, type_):
        raise ValueError("value needs to be an instance of type_")

    app.state.dependencies[type_] = value

def get_dependency(app: FastAPI, type_: Type[T]) -> T:
    value = app.state.dependencies[type_]
    if not isinstance(value, type_):
        raise ValueError(
            "The dependency that was found under the key type_ is not an instance of "
            "type_"
        )
    return value

Still, I think this should be an official feature.

With the above snippet, you can just inject it as a normal dependency (and you don't really need the function either)

def webserver_process(config: Configuration):
    ...

    app.dependency_overrides[Configuration] = lambda: config

    ....

@app.get('/')
def index(configuration: Configuration = Depends()):
    ...

@Mause this approach would inject the class as the API body  - Thats the main difference between your answer and what I understand @codethief was trying to achieve.
Is more similar to what FastAPI does for Request and the db session from the examples. Its not part of the API spec, its used to get the data / do business logic. As it stands the two are intertwined and you need to have a class that is not serializable.

I worked around this limitation by using Python's context variables.
A few lines of middleware will put them in place. The API endpoints can then either use some helper functions to get data from the context variables, possibly through fastapi.Depends.
It's possible for the entry point of the application, e.g., the main() to wrap the ASGI app in middleware like this:
from typing import Optional
from contextvars import ContextVar
from starlette.types import ASGIApp, Receive, Scope, Send

class ServerState:
    " Data to be exposed. Whatever would be global state otherwise. "

_server_state_var: ContextVar[Optional[ServerState]] = ContextVar("_server_state_var")

class ContextvarMiddleware:
    """
    Middleware that exposes the `ServerState` through contextvars to all
    underlying API endpoints.
    """
    def __init__(self, app: ASGIApp, server_state: ServerState) -> None:
        self.app = app
        self.server_state = server_state

    async def __call__(self, scope: Scope, receive: Receive, send: Send) -> None:
        _server_state_var.set(self.server_state)

        try:
            await self.app(scope, receive, send)
        finally:
            _server_state_var.set(None)

def create_api_handler(server_state: ServerState) -> ASGIApp:
    # To be called from "main()" or wherever the state is set.
    return ContextvarMiddleware(app, server_state=server_state)

With this, it's easy to define the FastAPI instance at the top level of a module and have it decoupled from any mutable state.

Hi @jonathanslenders , thanks so much for this interesting proposal!
To be honest, this is the first time I'm hearing of Python's contextvars module. Having now read the corresponding PEP 567, though, I'm getting the impression it's mainly meant for context-dependent state, i.e. state that's supposed to vary from execution context to execution context. In my case, however, I want to do the exact opposite: Namely, I'd like to inject the same global object / the same singleton into every execution context without having to actually resort to global state or the singleton pattern because it would obscure my application's true dependencies. While this would certainly work with contextvars, it requires the var to be set for every execution context separately which seems like a bit of overkill and, in case this is of any relevance, not very elegant.
PS: I believe in
    return ContextvarMiddleware(app, server_state=server_state)

you meant to write self.app?

is there any conclusion to this ticket?
What's the pattern recommended for injections outside of the request/app scope?


Chapter 10: Getting Into Digital Forensics

I can contribute to this chapter if the author is open to collaboration.  I wrote and taught 8+ university courses in eDisco/DFIR.  I've mentored several people who were completely switching careers into DFIR.  I inserted myself into the field 20+ years ago. Thx.

Of course! Please do!
-------- Original Message --------
On Jun 14, 2022, 17:05, awfr wrote:

I can contribute to this chapter if the author is open to collaboration. I wrote and taught 8+ university courses in eDisco/DFIR. I've mentored several people who were completely switching careers into DFIR. I inserted myself into the field 20+ years ago. Thx.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.Message ID: @.***>

Hey there! I'm tentatively setting 7/31/2022 as a milestone for publishing v1.0 of this book. We'll have the title decided in the next couple weeks which will be the first of multiple administrative tasks we'll complete in July. At this point, please let me know if you intend to have at least a working, editable version of your chapter by 7/31/2022.
If not, please know that's perfectly fine. It doesn't mean your chapter won't get published, it just won't get published in v1.0. It'll simply be added when it's ready to be published and I'll push out a new version of the book, (i.e., V1.3, v1.7, etc) with your new content. I hope we have about 10 ready to go by 7/31/2022 so we can push to publish v1.0 shortly thereafter, but I won't know that until I hear from you! So, please let me know!

Hey there, it's been a few months. Just wanted to check in and see how progress is going on your chapter? Whenever your chapter is ready to go, submit it to the repo, and once its reviewed it'll get published without delay! Cheers and hope you are well!


SomeFactory.build() fails with SubFactory on Django 1.8

Trying to .build() a factory will fail if the factory has a SubFactory. Normal call works. .build() also works on Django 1.7.7 (and factory_boy 2.5.1/2).
----> 1 PlacePartnerFactory.build()

/env/lib/python3.4/site-packages/factory/base.py in build(cls, **kwargs)
    539         """Build an instance of the associated class, with overriden attrs."""
    540         attrs = cls.attributes(create=False, extra=kwargs)
--> 541         return cls._generate(False, attrs)
    542
    543     @classmethod

/env/lib/python3.4/site-packages/factory/base.py in _generate(cls, create, attrs)
    482
    483         # Generate the object
--> 484         obj = cls._prepare(create, **attrs)
    485
    486         # Handle post-generation attributes

/env/lib/python3.4/site-packages/factory/base.py in _prepare(cls, create, **kwargs)
    459             return cls._create(model_class, *args, **kwargs)
    460         else:
--> 461             return cls._build(model_class, *args, **kwargs)
    462
    463     @classmethod

/env/lib/python3.4/site-packages/factory/base.py in _build(cls, model_class, *args, **kwargs)
    518             kwargs (dict): keyword arguments to use when building the class
    519         """
--> 520         return model_class(*args, **kwargs)
    521
    522     @classmethod

/env/lib/python3.4/site-packages/django/db/models/base.py in __init__(self, *args, **kwargs)
    466                 # "user_id") so that the object gets properly cached (and type
    467                 # checked) by the RelatedObjectDescriptor.
--> 468                 setattr(self, field.name, rel_obj)
    469             else:
    470                 setattr(self, field.attname, val)

/env/lib/python3.4/site-packages/django/db/models/fields/related.py in __set__(self, instance, value)
    666                     raise ValueError(
    667                         'Cannot assign "%r": "%s" instance isn\'t saved in the database.' %
--> 668                         (value, self.field.rel.to._meta.object_name)
    669                     )
    670                 setattr(instance, lh_field.attname, getattr(value, rh_field.attname))

ValueError: Cannot assign "Region(id=None, name='Poland', language_code='pl', country_code='')": "Region" instance isn't saved in the database.

Hi,
This is due to a behavior change in Django; see https://code.djangoproject.com/ticket/10811 and https://github.com/django/django/commit/5643a3b51be338196d0b292d5626ad43648448d3.
I've clarified the docs; unfortunately, I have no solution: .build() is broken when ForeignKey and OneToOneField are involved.

That's not good. What's the quickest way to get a dict from a database-saved factory? .values() or something else?

@rbarrois : I think we should re-open this issue to keep track of this django tracker ticket as it will most likely fix this issue.