Problem with Container Creation

Describe the issue
Hello,
I've tried to create a local development environment for Business Central using the BcContainerHelper.
But calling the New-BcContainer Command I get this error:
Import-TestToolkitToBcContainer Telemetry Correlation Id: xxxxxxxxxxxx
New-BcContainer Telemetry Correlation Id: xxxxxxxxxxxxx
Could not install the extension 'Permissions Mock' by 'Microsoft' (version '22.3.58346.59318') on tenant 'default' and company 'CRONUS DE' due to the following error: 'While opening company 'CRONUS DE', the following error occured: 'Sorry, the current
permissions prevented the action. (CodeUnit Master Data Management Execute: Base Application)'
Scripts used to create container and cause the issue
Install-Module BcContainerHelper -Force
Get-InstalledModule
Invoke-Expression (New-Object System.Net.WebClient).DownloadString("http://aka.ms/bcdockerscript")

$containerName = 'bc-dev'
$password = 'xxxxx'
$securePassword = ConvertTo-SecureString -String $password -AsPlainText -Force
$credential = New-Object pscredential 'admin', $securePassword
$auth = 'xxxxx'
$artifactUrl = Get-BcArtifactUrl -type 'Sandbox' -country 'de' -select 'Latest'
$licenseFile = 'xxxxxx'
New-BcContainer `
    -accept_eula `
    -containerName $containerName `
    -credential $credential `
    -auth $auth `
    -artifactUrl $artifactUrl `
    -imageName 'bc-dev' `
    -includeTestToolkit `
    -includePerformanceToolkit `
    -assignPremiumPlan `
    -licenseFile $licenseFile `
    -memoryLimit 16G `
    -vsixFile (Get-LatestAlLanguageExtensionUrl) `
    -updateHosts
Setup-BcContainerTestUsers -containerName $containerName -Password $credential.Password -credential $credential

Full output of scripts
Import-TestToolkitToBcContainer Telemetry Correlation Id: xxxxxxxxxxxx
New-BcContainer Telemetry Correlation Id: xxxxxxxxxxxxx
Could not install the extension 'Permissions Mock' by 'Microsoft' (version '22.3.58346.59318') on tenant 'default' and company 'CRONUS DE' due to the following error: 'While opening company 'CRONUS DE', the following error occured: 'Sorry, the current
permissions prevented the action. (CodeUnit Master Data Management Execute: Base Application)'
AL stack trace:
GlobalTriggerManagement(CodeUnit 49).OnAfterGetDatabaseTableTriggerSetup(Event) line 2 - Base Application by Microsoft
GlobalTriggerManagement(CodeUnit 49).GetDatabaseTableTriggerSetup line 8 - Base Application by Microsoft
"Global Triggers"(CodeUnit<PHONE_NUMBER>).GetDatabaseTableTriggerSetup(Event) line 2
"OData Initializer"(CodeUnit 1738).UpdateODataInitializedStatus line 17 - Base Application by Microsoft
"OData Initializer"(CodeUnit 1738).OnAfterLogin line 8 - Base Application by Microsoft
"System Initialization"(CodeUnit 150).OnAfterLogin(Event) line 2 - System Application by Microsoft
"System Initialization Impl."(CodeUnit 151).InitCompany line 5 - System Application by Microsoft
"Company Triggers"(CodeUnit<PHONE_NUMBER>).OnCompanyOpenCompleted(Event) line 2
' and AL stack trace:
GlobalTriggerManagement(CodeUnit 49).OnAfterGetDatabaseTableTriggerSetup(Event) line 2 - Base Application by Microsoft
GlobalTriggerManagement(CodeUnit 49).GetDatabaseTableTriggerSetup line 8 - Base Application by Microsoft
"Global Triggers"(CodeUnit<PHONE_NUMBER>).GetDatabaseTableTriggerSetup(Event) line 2
"OData Initializer"(CodeUnit 1738).UpdateODataInitializedStatus line 17 - Base Application by Microsoft
"OData Initializer"(CodeUnit 1738).OnAfterLogin line 8 - Base Application by Microsoft
"System Initialization"(CodeUnit 150).OnAfterLogin(Event) line 2 - System Application by Microsoft
"System Initialization Impl."(CodeUnit 151).InitCompany line 5 - System Application by Microsoft
"Company Triggers"(CodeUnit<PHONE_NUMBER>).OnCompanyOpenCompleted(Event) line 2
In C:\Program Files\WindowsPowerShell\Modules\BcContainerHelper\5.0.5\ContainerHandling\Invoke-ScriptInNavContainer.ps1:112 Zeichen:13
+             throw $errorMessage
+             ~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : OperationStopped: (Could not insta...Event) line 2
:String) [], RuntimeException
    + FullyQualifiedErrorId : Could not install the extension 'Permissions Mock' by 'Microsoft' (version '22.3.58346.59318') on tenant 'default' and company 'CRONUS DE' due to the following error: 'While opening company 'CRONUS DE', the following error occ
   ured: 'Sorry, the current permissions prevented the action. (CodeUnit Master Data Management Execute: Base Application)'
AL stack trace:
GlobalTriggerManagement(CodeUnit 49).OnAfterGetDatabaseTableTriggerSetup(Event) line 2 - Base Application by Microsoft
GlobalTriggerManagement(CodeUnit 49).GetDatabaseTableTriggerSetup line 8 - Base Application by Microsoft
"Global Triggers"(CodeUnit<PHONE_NUMBER>).GetDatabaseTableTriggerSetup(Event) line 2
"OData Initializer"(CodeUnit 1738).UpdateODataInitializedStatus line 17 - Base Application by Microsoft
"OData Initializer"(CodeUnit 1738).OnAfterLogin line 8 - Base Application by Microsoft
"System Initialization"(CodeUnit 150).OnAfterLogin(Event) line 2 - System Application by Microsoft
"System Initialization Impl."(CodeUnit 151).InitCompany line 5 - System Application by Microsoft
"Company Triggers"(CodeUnit<PHONE_NUMBER>).OnCompanyOpenCompleted(Event) line 2
' and AL stack trace:
GlobalTriggerManagement(CodeUnit 49).OnAfterGetDatabaseTableTriggerSetup(Event) line 2 - Base Application by Microsoft
GlobalTriggerManagement(CodeUnit 49).GetDatabaseTableTriggerSetup line 8 - Base Application by Microsoft
"Global Triggers"(CodeUnit<PHONE_NUMBER>).GetDatabaseTableTriggerSetup(Event) line 2
"OData Initializer"(CodeUnit 1738).UpdateODataInitializedStatus line 17 - Base Application by Microsoft
"OData Initializer"(CodeUnit 1738).OnAfterLogin line 8 - Base Application by Microsoft
"System Initialization"(CodeUnit 150).OnAfterLogin(Event) line 2 - System Application by Microsoft
"System Initialization Impl."(CodeUnit 151).InitCompany line 5 - System Application by Microsoft
"Company Triggers"(CodeUnit<PHONE_NUMBER>).OnCompanyOpenCompleted(Event) line 2

Additional context

My colleagues could use a similar script in the past and it worked. Now I tried to do it and now it won't

Thanks in advance!

It looks like your license file doesn't have enough permissions.
Try to remove the licenseFile parameter - the CRONUS license (default) should work.

Thank you @freddydk,
that was indeed the solution!


Update README.md

https://www.grammarly.com/blog/comma-before-and/
Needed my grammer-nazi fix for the day!  :)
p.s. nice tool.

Thanks for the corrections!


Fixed code for authentication using AzureCredentials

Just fixed method name..

Can one of the admins verify this patch?

Hi @mholec, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no contribution license agreement is required at this point. Real humans will now evaluate your PR.

TTYL, AZPRBOT;

@azuresdkci test this please

@hovsepm Please assign appropriately to someone on the fluent team to triage


View debug logs without having to shell via a verbosity variable in .plzconfig

Problem
Viewing debug logs in the Wheel Resolver requires going into --shell and manually passing --verbosity, which creates toil when debugging.
Proposal
Create a new Python Rules plugin to pass verbosity levels via .plzconfig.
Change in UX to get debug logs
Today:

You need to run the command with --shell (e.g. plz build third_party/python3:_PyInquirer#download --shell)
Copy the bash command when you enter the container (e.g. $TOOL --package {LIBRARY} --version {VERSION} --prereleases True and paste it to the input starting with $
Add --verbosity debug and then run it to get the logs

With this proposal:

Change verbosity in .plzconfig to debug
Run the build command (e.g. plz build third_party/python3:_{LIBRARY}#download
Change verbosity to info or another value once you want to leave debug mode

Graceful error handling
This proposal gracefully handles wrong values to verbosity and lists the valid values.
For example when setting in .plzconfig verbosity = debg the error message printed says
Error building target //third_party/python3:_{LIBRARY}#download: exit status 2
Usage: wheel_resolver.pex [OPTIONS]
Try 'wheel_resolver.pex --help' for help.

Error: Invalid value for '--verbosity' / '-v': Must be CRITICAL, ERROR, WARNING, INFO or DEBUG, not debg

Do you need to bump a/the version and update the changelog here?

Do you need to bump a/the version and update the changelog here?

I'll update the version in a future PR


[Timeline] Assign correct entity to errors thrown

This PR closes #133

:tada: This PR is included in version 1.15.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Let's get goma going

I'm trying to get turbo-cache working with goma. We're running a 512-core goma + buildbarn cluster with about a dozen builds per day. I got somewhere by just replacing buildbarn's CAS with turbo-cache, but would love to try running the whole thing on tc.
I've got a turbo-cache scheduler, cas and worker running but currently stuck on this error from goma:
exec call: error in check missing blobs: rpc error: code = Unimplemented desc = missing blobs: rpc error: code = Unimplemented
EDIT: got it to work! lmk if you want me to test with goma + chromium + lots of cores!

scheduler.json
{
  "stores": {
    "GRPC_LOCAL_STORE": {
      "grpc": {
        "instance_name": "local",
        "endpoints": ["grpc://goma.example.com:58981"],
        "store_type": "CAS"
      }
    },
    "GRPC_LOCAL_AC_STORE": {
      "grpc": {
        "instance_name": "local",
        "endpoints": ["grpc://goma.example.com:58981"],
        "store_type": "AC"
      }
    }
  },
  "schedulers": {
    "MAIN_SCHEDULER": {
      "simple": {
        "supported_platform_properties": {
          "OSFamily": "Exact"
        }
      }
    }
  },
  "servers": [{
    "listen_address": "<IP_ADDRESS>:58982",
    "services": {
      "ac": {
        "local": {
          "ac_store": "GRPC_LOCAL_AC_STORE"
        }
      },
      "execution": {
        "local": {
          "cas_store": "GRPC_LOCAL_STORE",
          "scheduler": "MAIN_SCHEDULER",
        }
      },
      "capabilities": {
        "local": {
          "remote_execution": {
            "scheduler": "MAIN_SCHEDULER",
          }
        }
      }
    }
  },
  {
    "listen_address": "<IP_ADDRESS>:58983",
    "services": {
      "worker_api": {
        "scheduler": "MAIN_SCHEDULER",
      }
    }
  }]
}

cas.json
{
  "stores": {
    "CAS_MAIN_STORE": {
      "memory": {
        "eviction_policy": {
          "max_bytes":<PHONE_NUMBER>0
        }
      }
    },
    "AC_MAIN_STORE": {
      "memory": {
        "eviction_policy": {
          "max_bytes":<PHONE_NUMBER>0
        }
      }
    }
  },
  "servers": [{
    "listen_address": "<IP_ADDRESS>:58981",
    "services": {
      "cas": {
        "local": {
          "cas_store": "CAS_MAIN_STORE"
        }
      },
      "ac": {
        "local": {
          "ac_store": "AC_MAIN_STORE"
        }
      },
      "capabilities": {
        "local": {}
      },
      "bytestream": {
        "cas_stores": {
          "local": "CAS_MAIN_STORE"
        },
        "max_bytes_per_stream": 64000
      }
    }
  }]
}

worker.json
{
  "stores": {
    "GRPC_LOCAL_STORE": {
      "grpc": {
        "instance_name": "local",
        "endpoints": ["grpc://goma.example.com:58981"],
        "store_type": "CAS"
      }
    },
    "GRPC_LOCAL_AC_STORE": {
      "grpc": {
        "instance_name": "local",
        "endpoints": ["grpc://goma.example.com:58981"],
        "store_type": "AC"
      }
    },
    "WORKER_FAST_SLOW_STORE": {
      "fast_slow": {
        "fast": {
          "filesystem": {
            "content_path": "~/.cache/turbo-cache/data-worker-test/content_path-cas",
            "temp_path": "~/.cache/turbo-cache/data-worker-test/tmp_path-cas",
            "eviction_policy": {
              "max_bytes":<PHONE_NUMBER>0
            }
          }
        },
        "slow": {
          "ref_store": {
            "name": "GRPC_LOCAL_STORE"
          }
        }
      }
    }
  },
  "workers": [{
    "local": {
      "worker_api_endpoint": {
        "uri": "grpc://goma.example.com:58983"
      },
      "cas_fast_slow_store": "WORKER_FAST_SLOW_STORE",
      "ac_store": "GRPC_LOCAL_AC_STORE",
      "work_directory": "~/.cache/turbo-cache/work",
      "platform_properties": {
        "OSFamily": {
          "values": ["Linux"]
        }
      }
    }
  }],
  "servers": []
}

Duh. Added the CAS into the scheduler config and am getting further!

Sorry, I got it all to work with the proper routing in scheduler.json.

I love the thought trail on this issue, it's just like mine :). I've been busy with other stuff, but I'll hopefully get round to finishing off the example Docker soon.  Out of interest, have you got a measure of how much better Turbo Cache is performing over your BuildBarn implementation?  Personally, I never got a functioning BuildBarn implementation to compare it to (I gave up before it was functional).

@chrisstaite-menlo so I haven't got any hard measurements, so take this with a grain of salt :)
The first build with a completely empty cache seemed to take a little bit longer and have lower TPS than with buildbarn, for some reason. However, after everything is in the CAS, the next build has somewhat better throughput and better overall time compared to buildbarn, although the difference is pretty small. I'm certainly not seeing a massive difference.
My setup is roughly as follows: dedicated 32C/64T Windows Server building with goma, with local SSD, plus 4 x 32C/64T (total 256 cores currently, sometimes 512 with cloud) worker machines, also with local SSD, each with 128GB mem. One of the worker machines is also goma-server and TC/BB scheduler and CAS. The goma machine is on a 10Gbps uplink, the others on 1Gbps, connected via VLAN (at Hetzner). I'm seeing traffic occasionally peak above 500Mbps to the Windows machine. I've also been increasing goma's concurrency parameters to minor effect (before TC).
Although a build time of 16 minutes is pretty great for a full Chromium build with a full CAS, I'm still left wondering where there could be bottlenecks. In the warm-cache case, neither the CPUs nor the network are saturated (using ninja -j1000)...  The full cache is about 69G.

It's in my to-do list to get the Goma example done. I have been busy with other things.
I'd be interested to research why the first build is slower, maybe the FUSE setup? I want to support a FUSE setup and native docker setup, but working with Investors lately, so my ability to work on this has been limited 😕

FYI a fuse setup paired with dedup store should be super fast for initial builds.

@allada I'm not using FUSE nor Docker, running everything straight on the OS; cross-compiling to Windows on Linux. So it's really quite straightforward. I'll continue to give both cases (TC/BB) more runs.

I had to patch the goma-server before it would cross-compile for me: https://github.com/kschzt/goma-server

Here are my current, working configs:
cas
{
  "stores": {
    "CAS_MAIN_STORE": {
      "fast_slow": {
        "fast": {
          "memory": {
            "eviction_policy": {
              "max_bytes":<PHONE_NUMBER>0
            }
          }
        },
        "slow": {
          "filesystem": {
            "content_path": "/home/rbe/rbe/tc/content_path-cas",
            "temp_path": "/home/rbe/rbe/tc/tmp_path-cas",
            "eviction_policy": {
              "max_bytes":<PHONE_NUMBER>00,
              "evict_bytes":<PHONE_NUMBER>0
            }
          }
        }
      }
    },

    "AC_MAIN_STORE": {
      "fast_slow": {
        "fast": {
          "memory": {
            "eviction_policy": {
              "max_bytes":<PHONE_NUMBER>0
            }
          }
        },
        "slow": {
           "filesystem": {
             "content_path": "/home/rbe/rbe/tc/content_path-cas_ac",
             "temp_path": "/home/rbe/rbe/tc/tmp_path-cas_ac",
             "eviction_policy": {
                "max_bytes":<PHONE_NUMBER>0,
             }
          }
        }
      }
    }
  },
  "servers": [{
    "listen_address": "<IP_ADDRESS>:58981",
      "services": {
        "cas": {
          "local": {
            "cas_store": "CAS_MAIN_STORE"
            }
        },
        "ac": {
          "local": {
          "ac_store": "AC_MAIN_STORE"
          }
      },
      "capabilities": {},
      "bytestream": {
        "cas_stores": {
          "local": "CAS_MAIN_STORE",
        },
        "max_bytes_per_stream": 64000, // 64kb.
      },
      "prometheus": {}
    }
  }]
}

scheduler
{
  "stores": {
    "GRPC_LOCAL_STORE": {
      "grpc": {
        "instance_name": "local",
        "endpoints": ["grpc://goma.example.com:58981"],
        "store_type": "CAS"
      }
    },
    "GRPC_LOCAL_AC_STORE": {
      "grpc": {
        "instance_name": "local",
        "endpoints": ["grpc://goma.example.com:58981"],
        "store_type": "AC"
      }
    }
  },
  "schedulers": {
    "MAIN_SCHEDULER": {
      "simple": {
        "supported_platform_properties": {
          "OSFamily": "Exact"
        }
      }
    }
  },
  "servers": [{
    "listen_address": "<IP_ADDRESS>:58982",
    "services": {
      "cas": {
        "local": {
          "cas_store": "GRPC_LOCAL_STORE"
        }
      },
      "ac": {
        "local": {
          "ac_store": "GRPC_LOCAL_AC_STORE"
        }
      },
      "bytestream": {
        "cas_stores": {
         "local": "GRPC_LOCAL_STORE"
        },
        "max_bytes_per_stream": 64000,
      },
      "execution": {
        "local": {
          "cas_store": "GRPC_LOCAL_STORE",
          "scheduler": "MAIN_SCHEDULER",
        }
      },
      "capabilities": {
        "local": {
          "remote_execution": {
            "scheduler": "MAIN_SCHEDULER",
          }
        }
      }
    }
  },
  {
    "listen_address": "<IP_ADDRESS>:58983",
    "services": {
      "worker_api": {
        "scheduler": "MAIN_SCHEDULER",
      }
    }
  }]
}```

worker

{
"stores": {
"GRPC_LOCAL_STORE": {
"grpc": {
"instance_name": "local",
"endpoints": ["grpc://goma.example.com:58981"],
"store_type": "CAS"
}
},
"GRPC_LOCAL_AC_STORE": {
"grpc": {
"instance_name": "local",
"endpoints": ["grpc://goma.example.com:58981"],
"store_type": "AC"
}
},
"WORKER_FAST_SLOW_STORE": {
"fast_slow": {
"fast": {
"filesystem": {
"content_path": "~/.cache/turbo-cache/data-worker-test/content_path-cas",
"temp_path": "~/.cache/turbo-cache/data-worker-test/tmp_path-cas",
"eviction_policy": {
"max_bytes":<PHONE_NUMBER>0
}
}
},
"slow": {
"ref_store": {
"name": "GRPC_LOCAL_STORE"
}
}
}
}
},
"workers": [{
"local": {
"worker_api_endpoint": {
"uri": "grpc://goma.example.com:58983"
},
"cas_fast_slow_store": "WORKER_FAST_SLOW_STORE",
"ac_store": "GRPC_LOCAL_AC_STORE",
"work_directory": "~/.cache/turbo-cache/work",
"platform_properties": {
"OSFamily": {
"values": ["Linux"]
}
}
}
}],
"servers": []
}

goma-server

./remoteexec_proxy
-port 5050
-remoteexec-addr goma.example.com:58982
-remote-instance-name local
-insecure-remoteexec
-service-account-json /home/ci/googlesa.json
-exec-config-file /home/ci/goma-config
-max-digest-cache-entries 200000000
-allowed-users<EMAIL_ADDRESS>-redis-max-active-conns 400
-redis-max-idle-conns 100

and the /home/ci/goma-config file something like this (mapping the compiler hashes for cross-compilation)

version_id:"1",
configs: [
{
dimensions: [ "os:win" ],
cmd_descriptor: {
selector: {
name: "clang-cl",
version: "clang version 16.0.0 (https://chromium.googlesource.com/a/external/github.com/llvm/llvm-project 60809cd29308e78d662375e6957a2b1b85415def)",
target: "x86_64-pc-windows-msvc",
binary_hash: "e583ccc29101c6ddfd286ad9c93db7bb240ad4f372db0a5aa946d8c73e40dbd6",
},
setup: {
cmd_file: {
path: "/home/goma/src/chromium/src/third_party/llvm-build/Release+Asserts/bin/clang-cl",
is_executable: true,
},
path_type: 1,
},
emulation_opts: {
respect_client_include_paths: true,
},
cross: {
clang_need_target: true,
windows_cross: true,
},
},
build_info: {
creator: "kschzt",
hostname: "<IP_ADDRESS>",
toolchain: "clang-cl",
},
remoteexec_platform: {
properties: [ { name: "OSFamily", value: "Linux" } ],
rbe_instance_basename: "local",
has_nsjail: false,
},
}
]
}

@allada Ahh, I think the lower throughput is because these workers saturate the CPU's more than buildbarn, which has a limited amount of workers (70 per 64 threads), so this takes load avg to 120+ where it should be below 64 - and then I see context deadline exceeded's. Maybe I should make a precondition_script to throttle workers?

@kschzt oh, you 100% need to add a property modifier to limit the concurrency to 1 per core:
  "schedulers": {
    "MAIN_SCHEDULER": {
      "property_modifier": {
        "modifications": [
          {"Add": {"name": "cpu_count", "value": "1"}},
          {"Remove": "container-image"},
          {"Remove": "InputRootAbsolutePath"}
        ],
        "scheduler": {
          "simple": {
            "supported_platform_properties": {
              "OSFamily": "Exact",
              "cpu_count": "Minimum"
            }
          }
        }
      }
    }
  },

And on the worker:
      "platform_properties": {
        "OSFamily": {
          "values": ["Linux"],
        },
        "cpu_count": {
          "query_cmd": "nproc"
        },
      },

I enabled the dedup store, the cpu_count stuff, and reduced -j to 512. It's better on the slow case, and set a new record of 13.5 mins in the warm-CAS case. Awesome!

In your case, I'd try dedup store into a compression store. The dedup store should excell on incremental changes because object files usually are mostly the same between builds, and dedup store will attempt to only download the changes.
You may also want to play with enabling gzip transport compression. You need to be careful here because gzip takes a lot of cpu time on the server and client.
Using grpc store is probably pretty sub-optimal because you can't take advantage of a lot of optimizations. When I get home, I'm going to see about making an extension of grpc store to support things like dedup and compression store.

Here's what I have for the CAS atm. I think I'm basically doing what you suggested?
I'll increase the memory store size a bit as I have some more to spare.
I've got all 5 machines in the same DC, and they're all dedicated boxes.
{
  "stores": {
    "FS_CONTENT_STORE": {
      "compression": {
        "compression_algorithm": {
          "LZ4": {}
        },
        "backend": {
          "filesystem": {
            "content_path": "/home/rbe/rbe/tc/content_path-cas",
            "temp_path": "/home/rbe/rbe/tc/tmp_path-cas",
            "eviction_policy": {
              "max_bytes":<PHONE_NUMBER>00, // 200gb
            }
          }
        }
      }
    },

    "CAS_MAIN_STORE": {
      "verify": {
        "backend": {
          "size_partitioning": {
            "size": 262144,
            "lower_store": {
              "ref_store": {
                "name": "FS_CONTENT_STORE"
              }
            },
            "upper_store": {
              "dedup": {
                "index_store": {
                  "fast_slow": {
                    "fast": {
                      "memory": {
                        "eviction_policy": {
                          "max_bytes":<PHONE_NUMBER>0, // 20gb
                        }
                      }
                    },
                    "slow": {
                      "filesystem": {
                        "content_path": "/home/rbe/rbe/tc/content_path-index",
                        "temp_path": "/home/rbe/rbe/tc/tmp_path-index",
                        "eviction_policy": {
                          "max_bytes":<PHONE_NUMBER>00, // 120gb
                        }
                      }
                    }
                  }
                },
                "content_store": {
                  "ref_store": {
                    "name": "FS_CONTENT_STORE"
                  }
                }
              }
            }
          }
        },
        "verify_size": true,
        "verify_hash": true
      }
    },

    "AC_MAIN_STORE": {
      "fast_slow": {
        "fast": {
          "memory": {
            "eviction_policy": {
              "max_bytes":<PHONE_NUMBER>0 // 10gb
            }
          }
        },
        "slow": {
           "filesystem": {
             "content_path": "/home/rbe/rbe/tc/content_path-cas_ac",
             "temp_path": "/home/rbe/rbe/tc/tmp_path-cas_ac",
             "eviction_policy": {
                "max_bytes":<PHONE_NUMBER>0, // 20gb
             }
          }
        }
      }
    }
  },
  "servers": [{
    "listen_address": "<IP_ADDRESS>:58981",
      "services": {
        "cas": {
          "local": {
            "cas_store": "CAS_MAIN_STORE"
            }
        },
        "ac": {
          "local": {
          "ac_store": "AC_MAIN_STORE"
          }
      },
      "capabilities": {},
      "bytestream": {
        "cas_stores": {
          "local": "CAS_MAIN_STORE",
        },
        "max_bytes_per_stream": 64000, // 64kb.
      },
      "prometheus": {}
    }
  }]
}

I'd suggest removing the verify store, unless you think that it's likely the files on disk will degrade.

Thanks a lot for your help gentlemen! Appreciate it. turbo-cache is working great now with goma. Better than buildbarn :) 🙏


Can not create confusion_matrix

This is my code for the network. I am using tensorflow 1.4 and I am using my nvidia for training. The code is working. I want to create a confusion matrix and I do it like this
with tf.device('/device:GPU:0'):
images = tf.placeholder(tf.float32, [None, IMAGE_WIDTH, IMAGE_LENGTH, 3])
labels = tf.placeholder(tf.int64, [None])
flat = tf.layers.flatten(labels)
logits = tf.layers.dense(flat, len(set(train_labels_arr)), tf.nn.relu)

predicted_labels = tf.argmax(tf.nn.softmax(logits), 1)

cross_entropy = tf.nn.sparse_softmax_cross_entropy_with_logits(logits = logits,
                                                               labels = labels)
loss = tf.reduce_mean(cross_entropy)

optimizer = tf.train.AdamOptimizer(learning_rate=0.001).minimize(loss)

correct_prediction = tf.equal(predicted_labels, labels)
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

init = tf.global_variables_initializer()

I want to create a confusion matrix and I do it like this:
confusion_matrix = tf.confusion_matrix(labels=labels, predictions=predicted_labels,
num_classes=len(set(train_labels_arr)),dtype=tf.int64)
When I call:
session = tf.Session()
session.run(init)
I got very weird error, but if remove the line with the confusion_matrix everything is ok: InvalidArgumentError: Cannot assign a device for operation 'confusion_matrix_4/assert_non_negative/assert_less_equal/Assert/AssertGuard/Assert': Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available.
[[Node: confusion_matrix_4/assert_non_negative/assert_less_equal/Assert/AssertGuard/Assert = Assert[T=[DT_STRING, DT_STRING, DT_STRING, DT_INT64], summarize=3, _device="/device:GPU:0"](confusion_matrix_4/assert_non_negative/assert_less_equal/Assert/AssertGuard/Assert/Switch, confusion_matrix_4/assert_non_negative/assert_less_equal/Assert/AssertGuard/Assert/data_0, confusion_matrix_4/assert_non_negative/assert_less_equal/Assert/AssertGuard/Assert/data_1, confusion_matrix_4/assert_non_negative/assert_less_equal/Assert/AssertGuard/Assert/data_2, confusion_matrix_4/assert_non_negative/assert_less_equal/Assert/AssertGuard/Assert/Switch_1)]]
Caused by op u'confusion_matrix_4/assert_non_negative/assert_less_equal/Assert/AssertGuard/Assert', defined at:
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/lachaka/tensorflow/lib/python2.7/site-packages/ipykernel_launcher.py", line 16, in
app.launch_new_instance()
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/traitlets/config/application.py", line 658, in launch_instance
app.start()
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/ipykernel/kernelapp.py", line 478, in start
self.io_loop.start()
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/zmq/eventloop/ioloop.py", line 177, in start
super(ZMQIOLoop, self).start()
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tornado/ioloop.py", line 888, in start
handler_func(fd_obj, events)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/zmq/eventloop/zmqstream.py", line 440, in _handle_events
self._handle_recv()
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/zmq/eventloop/zmqstream.py", line 472, in _handle_recv
self._run_callback(callback, msg)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/zmq/eventloop/zmqstream.py", line 414, in _run_callback
callback(*args, **kwargs)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/ipykernel/kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/ipykernel/kernelbase.py", line 233, in dispatch_shell
handler(stream, idents, msg)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/ipykernel/kernelbase.py", line 399, in execute_request
user_expressions, allow_stdin)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/ipykernel/ipkernel.py", line 208, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/ipykernel/zmqshell.py", line 537, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2718, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2822, in run_ast_nodes
if self.run_code(code, result):
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2882, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 41, in
num_classes=len(set(train_labels_arr)),dtype=tf.int64)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/confusion_matrix.py", line 162, in confusion_matrix
labels, message='labels contains negative values')],
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/check_ops.py", line 237, in assert_non_negative
return assert_less_equal(zero, x, data=data, summarize=summarize)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/check_ops.py", line 443, in assert_less_equal
return control_flow_ops.Assert(condition, data, summarize=summarize)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/util/tf_should_use.py", line 107, in wrapped
return _add_should_use_warning(fn(*args, **kwargs))
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 134, in Assert
condition, no_op, true_assert, name="AssertGuard")
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/util/deprecation.py", line 316, in new_func
return func(*args, **kwargs)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1864, in cond
orig_res_f, res_f = context_f.BuildCondBranch(false_fn)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1725, in BuildCondBranch
original_result = fn()
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 132, in true_assert
condition, data, summarize, name="Assert")
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_logging_ops.py", line 47, in _assert
name=name)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1470, in init
self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

Thank you for your post. We noticed you have not filled out the following field in the issue template. Could you update them if they are relevant in your case, or leave them as N/A? Thanks.
Have I written custom code
OS Platform and Distribution
TensorFlow installed from
TensorFlow version
Bazel version
CUDA/cuDNN version
GPU model and memory
Exact command to reproduce

Have I written custom code: Yes, I am trying to classify images
OS Platform and Distribution: Ubuntu 16.04 x64
TensorFlow installed from: pip
TensorFlow version: 1.4
Bazel version: N/A
CUDA/cuDNN version: 7.5
GPU model and memory: Nvidia 740M 2GB
RAM: 6GB
Exact commnad to reproduce: N/A I am using jupyter

@lachaka just to do a little fault isolation, if you place the confusion matrix calculation on CPU, does the code work?

Yeah, if I use CPU the is no error.

More sanity checking: it kinda looks like you're getting the weird failure because one of the assert operations isn't implemented on the GPU. But the assertion line that's blowing is at :162 of confusion_matrix.py:
https://github.com/tensorflow/tensorflow/blob/b2a0f1c45b2283910548ebd88ee5aaf4b6fc6077/tensorflow/python/ops/confusion_matrix.py#L162
Can you copy the assertion that labels and predictions are non-negative to just before your confusion matrix code, and see if it fires on GPU but not CPU? That would suggest that you're getting different calculated values on CPU and GPU, and maybe help us find the root cause.

InvalidArgumentError: Cannot assign a device for operation 'assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert': Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available.
[[Node: assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert = Assert[T=[DT_STRING, DT_STRING, DT_STRING, DT_INT64], summarize=3, _device="/device:GPU:0"](assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert/Switch, assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert/data_0, assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert/data_1, assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert/data_2, assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert/Switch_1)]]
Caused by op u'assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert', defined at:
File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/lachaka/tensorflow/lib/python2.7/site-packages/ipykernel_launcher.py", line 16, in
app.launch_new_instance()
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/traitlets/config/application.py", line 658, in launch_instance
app.start()
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/ipykernel/kernelapp.py", line 478, in start
self.io_loop.start()
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/zmq/eventloop/ioloop.py", line 177, in start
super(ZMQIOLoop, self).start()
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tornado/ioloop.py", line 888, in start
handler_func(fd_obj, events)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/zmq/eventloop/zmqstream.py", line 440, in _handle_events
self._handle_recv()
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/zmq/eventloop/zmqstream.py", line 472, in _handle_recv
self._run_callback(callback, msg)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/zmq/eventloop/zmqstream.py", line 414, in _run_callback
callback(*args, **kwargs)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tornado/stack_context.py", line 277, in null_wrapper
return fn(*args, **kwargs)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/ipykernel/kernelbase.py", line 283, in dispatcher
return self.dispatch_shell(stream, msg)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/ipykernel/kernelbase.py", line 233, in dispatch_shell
handler(stream, idents, msg)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/ipykernel/kernelbase.py", line 399, in execute_request
user_expressions, allow_stdin)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/ipykernel/ipkernel.py", line 208, in do_execute
res = shell.run_cell(code, store_history=store_history, silent=silent)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/ipykernel/zmqshell.py", line 537, in run_cell
return super(ZMQInteractiveShell, self).run_cell(*args, **kwargs)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2718, in run_cell
interactivity=interactivity, compiler=compiler, result=result)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2822, in run_ast_nodes
if self.run_code(code, result):
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2882, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "", line 39, in
labels, message='labels contains negative values')],
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/check_ops.py", line 237, in assert_non_negative
return assert_less_equal(zero, x, data=data, summarize=summarize)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/check_ops.py", line 443, in assert_less_equal
return control_flow_ops.Assert(condition, data, summarize=summarize)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/util/tf_should_use.py", line 107, in wrapped
return _add_should_use_warning(fn(*args, **kwargs))
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 134, in Assert
condition, no_op, true_assert, name="AssertGuard")
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/util/deprecation.py", line 316, in new_func
return func(*args, **kwargs)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1864, in cond
orig_res_f, res_f = context_f.BuildCondBranch(false_fn)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 1725, in BuildCondBranch
original_result = fn()
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 132, in true_assert
condition, data, summarize, name="Assert")
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/ops/gen_logging_ops.py", line 47, in _assert
name=name)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/home/lachaka/tensorflow/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1470, in init
self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access
InvalidArgumentError (see above for traceback): Cannot assign a device for operation 'assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert': Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available.
[[Node: assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert = Assert[T=[DT_STRING, DT_STRING, DT_STRING, DT_INT64], summarize=3, _device="/device:GPU:0"](assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert/Switch, assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert/data_0, assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert/data_1, assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert/data_2, assert_non_negative_1/assert_less_equal/Assert/AssertGuard/Assert/Switch_1)]]

OK, so it looks like the assert itself, running on the GPU, is what's not implemented.
Another debugging step: can you run the assert on the CPU (so we know it works), but generate the predictions and labels on the GPU, so we can tell whether the assert is correctly firing based on funky GPU-generated data, or whether the data is good, and it's just the assertion that needs to be implemented on GPU (or mapped back to CPU, which TF should have done).

Well, I execute the tf.confusion_matrix on CPU and It works. It creates the matrix. The other operations like generating predictions and so on are calculated by the GPU.

Sounds like the values generated by GPU are good, but some part of the assertion isn't implemented on GPU.


hope support mobilenetv3 small

hello Floe,
Could you tell me how to add the support of mobilenetv3 small?

Not quite sure what you mean, deeplab v3 is already based on mobilenet v3? Or do you have a link for the specific network you want to support?

FYI
https://ai.googleblog.com/2020/10/background-features-in-google-meet.html

FYI
https://ai.googleblog.com/2020/10/background-features-in-google-meet.html

Ah, interesting. First thing you would need is the actual model file in .tflite format, apparently it's not available in https://github.com/google/mediapipe/tree/master/mediapipe/models ?

Ah, interesting. First thing you would need is the actual model file in .tflite format, apparently it's not available in https://github.com/google/mediapipe/tree/master/mediapipe/models ?

Mobilnetv3 Large?
https://tfhub.dev/sayakpaul/lite-model/deeplabv3-mobilenetv3-cityscapes/1/default/2

Mobilnetv3 Large?
https://tfhub.dev/sayakpaul/lite-model/deeplabv3-mobilenetv3-cityscapes/1/default/2

Deeplab v3 is exactly what my project here already uses: https://github.com/floe/deepbacksub/blob/master/deeplabv3_257_mv_gpu.tflite
The only difference I can see is that the one you just posted is trained on Cityscapes, which won't help a lot with person segmentation.
The one from Google Meet would be much more interesting, because it is specifically optimized for person segmentation, but you would need to find the specific .tflite file for that Google Meet network.

Deeplab v3 is exactly what my project here already uses: https://github.com/floe/deepbacksub/blob/master/deeplabv3_257_mv_gpu.tflite
The only difference I can see is that the one you just posted is trained on Cityscapes, which won't help a lot with person segmentation.
The one from Google Meet would be much more interesting, because it is specifically optimized for person segmentation, but you would need to find the specific .tflite file for that Google Meet network.

Floe,
Thanks, Hope lucky to get that tfilte. Do you have idea about joint bilateral filter?

Floe,
Thanks, Hope lucky to get that tfilte. Do you have idea about joint bilateral filter?

Right now, it seems you can download the models from https://meet.google.com//rtcvidproc/release_1wttl/345264209/segm_full_v679.tflite and https://meet.google.com//rtcvidproc/release_1wttl/345264209/segm_lite_v681.tflite - and according to the model card they should be licensed with Apache 2.0, so you should in theory be allowed to re-use them. I'll give it a try and see if these can be integrated.

Right now, it seems you can download the models from https://meet.google.com//rtcvidproc/release_1wttl/345264209/segm_full_v679.tflite and https://meet.google.com//rtcvidproc/release_1wttl/345264209/segm_lite_v681.tflite - and according to the model card they should be licensed with Apache 2.0, so you should in theory be allowed to re-use them. I'll give it a try and see if these can be integrated.

Floe,
Great! Thanks a lot.

Floe,
Great! Thanks a lot.

Hello Floe,
I tried the new model, but the AllocateTensors() failed. I guess that the xnnpack need to be included.

Hello Floe,
I tried the new model, but the AllocateTensors() failed. I guess that the xnnpack need to be included.

Got  something may be useful
https://github.com/tensorflow/tfjs/issues/4177

Got  something may be useful
https://github.com/tensorflow/tfjs/issues/4177

I've just copied the relevant custom op from MediaPipe for now. See latest commit (https://github.com/floe/deepbacksub/commit/306da1b8869f75874a600cad3352ee11cc6071a3) for a implementation that's somewhat working (still some sort of synchronization error between capture thread and network evaluation happening here, though).

I've just copied the relevant custom op from MediaPipe for now. See latest commit (https://github.com/floe/deepbacksub/commit/306da1b8869f75874a600cad3352ee11cc6071a3) for a implementation that's somewhat working (still some sort of synchronization error between capture thread and network evaluation happening here, though).

Great! I will try.

Great! I will try.

I'm closing this for now, thanks for pointing me to the Google Meet segmentation model. Please open a new issue if something is still wonky.

I'm closing this for now, thanks for pointing me to the Google Meet segmentation model. Please open a new issue if something is still wonky.

Hello, I am trying to run google meet model on windows. But I still don't have any idea how to integrate the custom op from mediapipe. I cannot load the model using normal runtime because it doesn't have the custom operation.

You need transpose_conv_bias.* from https://github.com/google/mediapipe/tree/master/mediapipe/util/tflite/operations and register them as in https://github.com/floe/deepbacksub/blob/master/deepseg.cc#L252-L258.

Mediapipe officially released selfie segmentation support. You might want to update your app.
https://github.com/google/mediapipe/releases/tag/v0.8.5


Does graphkit works on pandas dataframe ?

I have pyspark data frame and I wanted to use graphkit inside pandas udf. Does graphkit has this support ? If so any documentation available.
Thanks

Graphticis is pure Python, it should work.

Any examples or documentation or sample code available on this ?

I was expecting it would be the other way round.
This library the only thing it can do is compose and executed great of functions, as exemplified in the README.


feat: report iframe target types

This is necessary so that we (and our users) can diagnose OOPIFs
and OOPIF-related issues.
Add a test to make sure that --site-per-process chromium flag actually forces iframes to be out-of-process.

Closing in favor of https://github.com/GoogleChrome/puppeteer/pull/4516


Adds createFragmentToCreateMock transform

I've also made some temporary changes to how arguments are parsed and passed in the index file. This is really just temporary, my end goal here is to add yargs and create more powerful argument logic. The benefit to that is ideally each transform can specify its own possible CLI flags in cases where you want a transform to take input.
Example in this case this transform is used as such:
yarn transform createFragmentToCreateMock '../hw-admin/app/javascript/**/*.spec.tsx' createPractitionerFragment Practitioner
Ideally this could eventually become:
yarn transform createFragmentToCreateMock '../hw-admin/app/javascript/**/*.spec.tsx' --createFragmentFunction createPractitionerFragment --typeName Practitioner
etc.

In terms of testing this, I've run it against hw-admin and converted, createPatientFragment, createPractitionerFragment, createVariantFragment, createBrandFragment and in each case this transform was producing output as expected.


Support for apple silicon

Awesome project! I want to test but I don't have a cuda gpu. What can I do if I want run locally using other gpu? Thanks
Maybe MPS or I don't know I am a newbie.

Hi there thanks a lot for using us & so glad unsloth was useful! There was already an issue opened for Apple Silicon support: https://github.com/unslothai/unsloth/issues/4

Thanks!


After your latest log of your trail, you can't re-watch your 10 results

Describe the bug
As mentioned in the application you'll get 10 free logs. The issue is that you can't re-watch your logs that you made because of the popup model that asks to activate your license.
Versions
Ray version <IP_ADDRESS>:
Composer: spatie/laravel-ray                 1.3.5              Easily debug Laravel apps
PHP version: PHP 8.0.1 (cli)
Laravel version:  v8.21.0
Steps to reproduce the behavior:

start a trail
Execute 9 ray logs
The 10th log is a JSON
See error

Expected behavior
A smaller popup alert in the top corners that warns you that you need to activate your license or otherwise you can't use ray in the further progress of your project.
Screenshots

Desktop (please complete the following information):

OS: Windows 10 Home 64-bit
Version: Build 19041

This is expected behaviour. To get rid of this, you should get a license.

This is expected behaviour. To get rid of this, you should get a license.


'Seed' is reset for each repeat.

Algorithm parameters on simple tasks should only be set once, even if they're inside a repeated task.  This is generally true, but particularly problematic for the 'seed' of a stochastic simulation inside a repeated task:  if it's set every time, you end up with just the single random run, repeated identically each time.

Fixed with https://github.com/sys-bio/tellurium/commit/0b4969463b60819206fdf08c285830b15510db07 for now:  still need to work on nested simulations.


Added Keys query operator

Per https://parse.com/docs/rest#queries-constraints, queries can now ask for only certain fields. Added relevant test, tidied up the .gitignore and added example to readme.

Also changed readme badges to use shields.io which makes them nice and pretty.

woohoo! quiet and .keys are in master.

@Xavdidtheshadow did you check if the cassette is still valid after a year :) ?

Sorry, I just noticed you updated it. Thanks!


Use Mermaid to render federated pipeline architecture

Imitate this: https://github.com/Azure/shrike/pull/357 .
@ShizeSu06 + @thomasp-ms — not urgent, but worth thinking about.

Closing this old issue for now, we have so few schemas that this would not bring too much value. Also, we can reconsider once we know where the official FL native support documentation will be hosted.


Long-Running jobs

Looking for an alternative to the abandonware that is "Elastic Database Jobs" in Azure.  It would be good to have some guidance as to which of the compared serverless services is most appropriate for long running database maintenance jobs.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 82ee9751-961d-7109-63d6-813b1862c632
Version Independent ID: 583d1b2f-9c1a-3580-9d14-37bb551eaf44
Content: Integration and automation platform options in Azure
Content Source: articles/azure-functions/functions-compare-logic-apps-ms-flow-webjobs.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

@SgtSpamCan Thank you for your feedback. I'll look into this from the Azure Functions side (since you have opened this issue under this doc) and share my findings on what's possible.
cc: @Mike-Ubezzi-MSFT do you have any thoughts on this matter?

@SgtSpamCan In my review, I haven't come across or found any remark or a statement saying that the "Elastic Database Jobs" is being deprecated or abandoned. Could you reference any links? If it is, indeed, being deprecated, there should certainly be a recommendation on the alternative and it'd be best to create a new issue under one of the docs for Elastic Database Jobs instead, and I'd be happy to assist and connect you with the right team.
As for your question around which of the serverless services discussed in this documentation, they are all capable of orchestrating and running long-running jobs. The distinction can be drawn more on how you want to implement your solution and to what depth and complexity.
Since this channel is more for driving improvements towards MS Docs and we haven't determined any changes to this particular doc based on our review of your feedback, we will now proceed to close this thread for now. If there are further questions regarding this matter, please reopen it and we will gladly continue the discussion.

I'll reference that the service itself is still in "Preview", and has been
for several years.
On Tue, Feb 18, 2020 at 1:42 PM Mike Urnun<EMAIL_ADDRESS>wrote:

@SgtSpamCan https://github.com/SgtSpamCan In my review, I haven't come
across or found any remark or a statement saying that the "Elastic
Database Jobs
https://docs.microsoft.com/en-us/azure/sql-database/elastic-jobs-overview"
is being deprecated or abandoned. Could you reference any links? If it is,
indeed, being deprecated, there should certainly be a recommendation on the
alternative and it'd be best to create a new issue under one of the docs
for Elastic Database Jobs instead, and I'd be happy to assist and connect
you with the right team.
As for your question around which of the serverless services discussed in
this documentation, they are all capable of orchestrating and running
long-running jobs. The distinction can be drawn more on how you want to
implement your solution and to what depth and complexity.
Since this channel is more for driving improvements towards MS Docs and we
haven't determined any changes to this particular doc based on our review
of your feedback, we will now proceed to close this thread for now. If
there are further questions regarding this matter, please reopen it and we
will gladly continue the discussion.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MicrosoftDocs/azure-docs/issues/47963?email_source=notifications&email_token=AFIFY5NE3UYYNR7OBMDOIULRDRB2RA5CNFSM4KT7CRW2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMEVDTQ#issuecomment-587813326,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFIFY5LE7VZRT7OYHEWPDZTRDRB2RANCNFSM4KT7CRWQ
.


Nginx got 400 Bad Request

Hardware: Raspberry Pi 5 8GB RAM
OS: Ubuntu server 20.04 LTS
python -m waifu2x.web --bind-addr <IP_ADDRESS> --port 8812 works fine on my machine.
After successfully loaded main page on my Nginx server, I got 400 Bad Request when I tried to upscale the pic:
400 Bad Request
Request Header Or Cookie Too Large
nginx/1.24.0 (Ubuntu)

And the error.log was clean. I have tried to enable larger size of client header buffers for the page, but it still didn't work.
Here is my page configuration file:
map $http_accept_language $lang {
  default en;
  ~ja ja;
  ~ru ru;
  ...
  ~en en;
}

#limit_req_zone $limit_req_key  zone=one:128m   rate=10r/m;
server  {
        listen 1002;
        server_name <IP_ADDRESS>;

        access_log	/var/log/nginx/waifu2x-access.log;
        error_log     /var/log/nginx/waifu2x-error.log;

        root   /home/metasequoiani/server/nunif-master/waifu2x/web/public_html;

        try_files $uri $uri/index.$lang.html $uri/index.html @api;

        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP  $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_read_timeout 160;

        client_max_body_size 64M;
        client_header_buffer_size 4k;
        large_client_header_buffers 4 16k;

        location ~ ^/(apple-touch-icon|browserconfig|mstile)(.*)\.(png|xml|ico)$ {
            return 204;
        }
        location @api {
            #limit_req zone=one burst=3;
	        set $limit_req_key $binary_remote_addr;
	        if ($request_uri = "/recaptcha_state.json") {
	            set $limit_req_key "";
	        }
            proxy_pass http://<IP_ADDRESS>:1002;
        }
        listen 80;
}

two listen ports listen 80; and listen 1002; are specified,
and proxy_pass http://<IP_ADDRESS>:1002; can cause infinite loop for listen 1002;. is it not http://localhost:8812/ ?

server_name <IP_ADDRESS>;

Perhaps this does not work. If you suspect that is the cause, try server_name _ ;

nginx on https://waifu2x.net/ I just add
keepalive_timeout 65;
client_max_body_size 5M;

to /etc/nginx.conf http {} context.
server config is almost the same as https://github.com/nagadomi/nunif/blob/master/waifu2x/web/appendix/waifu2x.nginx.conf

Hi, thank you for your advise. I'm freshman in network but fortunately it finally worked. I have to set longer survival time because waifu2x takes longer time to run the task on my Pi.
And I've also deployed unlimited version on Nginx.  Great thanks to your work!
Wish you a nice day :)


getShortestPaths & getAdjacencyMap inference failing

Not sure why - maybe because of recent TS upgrade (?), but few things in tests don't compile because they have trouble with proper type inference.
Mainly TEvent can't be inferred from concrete StateNode/StateMachine used in argument position.
Here:
https://github.com/davidkpiano/xstate/blob/9bdfe6d496ba81ef2ec0c17502fe9355565aaeee/packages/xstate-graph/test/graph.test.ts#L305
And here:
https://github.com/davidkpiano/xstate/blob/9bdfe6d496ba81ef2ec0c17502fe9355565aaeee/packages/xstate-graph/test/graph.test.ts#L171

Closing as stale - type inference is correct and tests are passing 😄


Make email none-case-sensitive

When creating/signing up a user with the same email but upper and lowercase (e.g<EMAIL_ADDRESS>and<EMAIL_ADDRESS>your crate creates two different users. As far as i know emails are none-case-sensitive. I fixed this by applying .to_lowercase() to the email, but i think it would be good to warn for this in the documentation or to apply this in your users.create_user() and signup() function

E-mail addresses can't be assumed to be case-insensitive; only the domain name portion of an e-mail address is case-insensitive. See https://beesbuzz.biz/code/439-Falsehoods-programmers-believe-about-email


[Notice] Plan to convert code base from Javascript to Typescript

It's in my mind for months.
We will be benifit as the following:

enable circular denpendency between modules
better async coding style: async/await (now we use co module)
strong type defination/checking for data structure
auto completion/static syntax check with vscode ide
static variable with initializing inside class
public/private setting for methods/member variables
other benifites...

after converted, there will be different if you:

use the master code base directly. solution: use ts-node instead of node, and write your code with typescript.
study typescript and setup typescript environment.

there will be no different if you:

use npm module to require('wechaty'). you can use javascript as before, becasuse typescript will be compiled to javascript before publish.

The lastest javascript version is: v0.4.0
Reference

The Future of Declaration Files
What's new in TypeScript

A nice slide about TypeScript
http://git.oschina.net/rockq/node-party/blob/master/archieves/5th/Angular_TypeScript.pdf

https://github.com/wechaty/wechaty/commit/b54bfbafbe0f64f80b1f9937635278319d84a66a
https://github.com/wechaty/wechaty/commit/dec557b8b79b967e76c884419d178e929b17f162
https://github.com/wechaty/wechaty/commit/12d491d170dccebeb108bb6fdae0f59b353cb795
https://github.com/wechaty/wechaty/commit/3917e703c7a330b5a9427c87810cd29cd2af50cf
https://github.com/wechaty/wechaty/commit/aa98e45a55fe1c623af6360d3ccd2e27daefb4b4
https://github.com/wechaty/wechaty/commit/5787f6dad515cbb304b19e0414b99fe734657e2c
https://github.com/wechaty/wechaty/commit/195d11eefd6befaf4c79e595687dd0a6717509ab

almost last step: tsc with @types https://github.com/Microsoft/TypeScript/issues/11491

this is caused by a bug of typescript : https://github.com/Microsoft/TypeScript/issues/11103 , could be fixed by add --moduleResolution node

stucked with a strange behaviour of ts-node : https://github.com/TypeStrong/ts-node/issues/216

unit tests need to convert, too...

Could Typescript save us from circular denpendence module undefined problem? https://github.com/Microsoft/TypeScript/issues/11576

cheers!

all done!

today, all tests passed in CI , and with docker!
cheers!

Due to export default can lead to problems, delete all the export default in wechaty.
so from now, we must use import { Wechaty } from 'wechaty' instead of import Wechaty from 'wechaty'.

Prepare send TypeScript Support Pull Request to AVA:
https://github.com/avajs/ava/issues/1109


Organization and Project pages

Implement functionality and layout based on new contract methods

[x]  Organization page
https://www.figma.com/file/3UUMChv0qaOOkHoddpEFgA/SOURC3?node-id=3295%3A672

https://www.figma.com/file/3UUMChv0qaOOkHoddpEFgA/SOURC3?node-id=3278%3A880

[x] Project page
https://www.figma.com/file/3UUMChv0qaOOkHoddpEFgA/SOURC3?node-id=3348%3A508

[ ] Organization edit form
https://www.figma.com/file/3UUMChv0qaOOkHoddpEFgA/SOURC3?node-id=2077%3A1216

[ ] member page
https://www.figma.com/file/3UUMChv0qaOOkHoddpEFgA/SOURC3?node-id=3281%3A1044

https://www.figma.com/file/3UUMChv0qaOOkHoddpEFgA/SOURC3?node-id=3284%3A463

[ ] project edit form
https://www.figma.com/file/3UUMChv0qaOOkHoddpEFgA/SOURC3?node-id=3348%3A429

While creating a project in the organization, when the transaction passed, the user should be redirected to a page with a list of projects of the current organization.

While creating a repo in the project, when the transaction passed, the user should be redirected to a page with a list of repos of the current project.

While adding members to the project, when the transaction passed, the user should be redirected to a page with a list of members of the current project.


Feature Request: Add option to hide OSD messages from script

I often find the messages distracting, adding a flag for so that it can be tweaked on the script opts conf would be awesome!

Sure why not.
By the way, did you solve the SQLite problem?  I saw the issue, but after a few minutes you closed it.

By the way, did you solve the SQLite problem? I saw the issue, but after a few minutes you closed it.

Oh sorry lol I put actually put it on the app repository by mistake, not on this one so closed it lol. I reissued it out here.
Sorry for the confusion. For now i just disabled the database using uselocaldb=0 and it works.
And thank you so much for replying so fast and for also for listening to me!

1.0.6 includes this feature, forgot to close the issue.


Building/running/closing app issue

If I use the pre-built binaries from the installer the open3mod.exe process closes but not when I build from source and load a model.  This is strange.  I am building the .NET 3.5 "Debug" and "Release" configurations.  Apart from a Devil copy error it builds and runs fine.
When I run the compiled version either Debug or Release and I do NOT open a model then the process closes fine.  But once I open model, even one of your Sample Scenes, the open3mod.exe process stays active when you close the app using the window red "x" button or File - Quit.  You can verify this in the Task Manager.  You can then launch multiple instances of open3mod.exe, all staying active if you load a model and then close.  Again your pre-built version does not do this.  I will continue to trace this but first I have to get familiar with C# again!  :)  I am a professional C++ developer and on occasion use C#.
Here is that error btw.
[quote]Error	2	The command "copy  C:\Dev\area51\Character\Open3D Viewer\open3mod-master\libs\devil-net\DevILNet\..\Libs\x86\DevIL.dll ......\open3mod\bin\Release
copy C:\Dev\area51\Character\Open3D Viewer\open3mod-master\libs\devil-net\DevILNet\..\Libs\x86\ILU.dll ......\open3mod\bin\Release" exited with code 1.	DevILNet
Warning	3	The field 'open3mod.FpsCameraController._rollAngle' is assigned but its value is never used	C:\Dev\area51\Character\Open3D Viewer\open3mod-master\open3mod\FpsCameraController.cs	45	23	open3mod
Warning	1	The predefined type 'System.Runtime.CompilerServices.ExtensionAttribute' is defined in multiple assemblies in the global alias; using definition from 'c:\Dev\area51\Character\Open3D Viewer\open3mod-master\libs\Mono.Cecil\Net20\Mono.Cecil.dll'	C:\Dev\area51\Character\Open3D Viewer\open3mod-master\libs\assimp-net\AssimpNet.Interop.Generator\CSC	AssimpNet.Interop.Generator[/quote]
Running on Win7 x64 and compiling with Visual Studio 2013 w/Update 3.
-Steve

The current prebuilt standalone version v1.1 on windows 7 64bit has the same issue.
Once you open an (.obj) file, the process continues to run after closing the window. It even keeps the file locked, so you can't move or delete it.

Reproduced. This is critical so I'll try to get a fix out asap.

Resolved by 1ec6ba1c1bb2afde569 but I still have to update the binaries.


Missing blocks in motion-compensated frame

Hi, Thank you for developing this great tool for motion extraction.
I used the tool to extract the motion vectors in the example video. Let's say we have the reference frame, the current frame, and the motion vectors corresponding to current frame.
I used the motion vectors to apply motion compensation on the reference frame in order to reconstruct the current frame. After motion compensation, there are some empty macro blocks left in the frame. They are the macro blocks which no motion vector was pointed to in the current frame. I suppose that for these macro blocks, no similar macro block in the reference frame was found.
Now I have two questions. Is there anywhere in the code that I can set the threshold for finding the similar Macroblock. I kind of hope to have less empty Macroblocks in the motion compensated frames. My second question is that is it possible that we have bi-directional frames using this tool? I think if we have bi-directional frames, there would be less empty macrobloacks in the motion-compensated frame.
I appreciate your response. Thank you!
Reference frame:

Current frame:

motion-compensated reference frame:

Hey zohrehazizi,
glad to hear that you find mv-extractor useful. To answer your questions:

Is there anywhere in the code that I can set the threshold for finding the similar Macroblock. I kind of hope to have less empty Macroblocks in the motion compensated frames.

-> This has to be done during encoding of the video using e.g. FFMPEG. With FFMPEG you can either transcode the video or split it up into individual frames and encode them as a video again. I am not sure if the codec directly supports a threshold as you need it. But I assume the compression level and profile (baseline, main, high) has an impact on the number of fully coded (i.e. not motion-compensated) macroblocks.

My second question is that is it possible that we have bi-directional frames using this tool? I think if we have bi-directional frames, there would be less empty macrobloacks in the motion-compensated frame.

Yes, B-frames are supported. But again, creating B-frames is part of the encoding procedure. The H.264 profile determines whether b-frames are available.


auto format all the things

I wanted to make sure CI passed on the auto format, since at one point it was removing trailing white space that was syntactically needed for some file types


07/23/2018 02:58 : Create Pull Request for branch master -> prTesting1_2018072302

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

split/updateTestCon1.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Integrate Inventory2 Storage with Forseti

Thanks for opening a Pull Request!
Here's a handy checklist to ensure your PR goes smoothly.

[ ] I signed Google's Contributor License Agreement
[ ] My code conforms to Google's python style.
[ ] My PR at a minimum doesn't decrease unit-test coverage (if applicable).
[ ] My PR has been functionally tested.
[ ] All of the unit-tests still pass.
[ ] Running pylint --rcfile=pylintrc passes.

These guidelines and more can be found in our contributing guidelines.

This is just a dry-run.


PyTorch Support

Is there PyTorch support for Lucid, or a similar library anyone could recommend?

I'm afraid there isn't any support for Pytorch. All the concepts should transfer, though, if you plan to build it!
I do know of https://github.com/sunalbert/lucid.pytorch, but I'm unsure what state this project is in. It does link to a bunch of projects in the readme, though I can't vouch for any of them. If you find one you'd recommend, please comment here and we may add it to our Readme.

One thing to note is that you may be able to convert your trained PyTorch model to TensorFlow using one of the model conversion libraries. If you do that, you could use lucid to visualize it. :)

Thanks, model conversion definitely seems like the route to go!

Did the model conversion route work ?

OpenAI announced a move to pytorch: https://openai.com/blog/openai-pytorch.
Any change of plans here?

Not presently for Lucid. PyTorch has several interpretability projects already who we've been talking to and sharing some of our (not very mature) PyTorch code with. Captum seems to have dedicated engineers working on user-ready infrastructure.
These projects are much better positioned than the lucid maintainers for maintaining a pytorch interpretability library. I'm hopeful  they'll support many of the same features eventually.

Since this is a top hit when Googling "lucid pytorch", just wanted to spread the word about the Lucent library! Essentially Lucid for PyTorch and works with torchvision.models. Check out the Colab notebooks at Lucent that reproduces the results from Lucid!


Force Right Click on certain NPCs

Request from Discord:

... NPC Right Click Only Filter ... Some monsters I find you want to have it default set to right click only (KQ, Nechs Bursting, Ardy Knights etc.) But swapping that option might get tedious ...

Could potentially be controlled like Tagging currently is (Shift + Right Click)

Jagex's recent statement prohibits Allows you to prioritise the walk-here option by holding a certain key, or similar and Removal or swapping of menu entries on any entity  during bosses


Active storage variants in Rails 6>

Since upgrading to Rails 6, my team changed to use variants in active_storage with vips. In vips the old:
resize: "100x100"
results in a broken file for rails_admin users that uses vips.
Instead I've added a check to see if the project is using ruby-vips and in that case they will be using the resize method:
resize_to_limit: [100, 100]
Another way to go to check it, is to check:
config.active_storage.variant_processor
if that is preferrable.

Merged in, thanks!


Allow dashboard to filter result by "language family"

Summary
One feature I feel strongly about including is the ability to filter the dashboard by language family. As a maintainer/contributor on Paketo Buildpacks it's easy for me to find what I'm looking for, but it might not be so clear to an outside user which buildpacks they should be looking at. I feel that being able to filter which buildpacks show up could make the user experience more straightforward.
cc/ @sophiewigmore

@sophiewigmore I see that you have assigned yourself to this issue - would you like any assistance?
I would love to see this feature implemented and I'd love to build on any existing work you've done.

@robdimsdale I was working on this a while ago and it kind of fell through the cracks. I'd love to see you pick this back up and I can help in any way. The branch with my changes is here: https://github.com/paketo-buildpacks/dashboard/tree/filter-by-topics.


Doesn't work with Browserify (double bug)

When I do
var localforage = require("localforage")
```

I got this error

```
Error: module "./drivers/indexeddb" not found from "/myproject/node_modules/localforage/dist/localforage.js"
```
Normal behavior path is not good.
As a workround I've tried

```js
var localforage = require("localforage/src/localforage")
```

But got another error with a missing

---

```
Error: module "promise" not found from "/myproject/node_modules/localforage/src/drivers/websql.js"
```

"es6-promise" is not declared as an npm dependency & it should to be usable via npm

As a final woraround, I've done

```
npm i -D es6-promise
ln -s ../node_modules/es6-promise node_modules/promise
```

And now it's working. This should be improved because having to remap promise to es6-promise this way is seriously ugly :D

dang, this is still broken with browserify

@mshwery After summing up the recent #412 & #413, you might want to take a look at how open-flight-planner is configured to use localforage. This is working both with the latest npm version and the latest master source.

@thgreasi thanks for the extra info. I checked those out but their setup really isn't different than my use case, other than the fact that i'm using a separate bundle. I'm still bundling the same way, and importing the same way.
I left more details here: https://github.com/mozilla/localForage/issues/412#issuecomment-135049022


fix: adjust scrollbar position in browser panes

In browser panes, the scrollbar position overlaps the pane separator.
This fits nicely with defaults symbols.
However, for a more minimalist approach with empty symbols, eg.
scrollbar: (symbols: ["", "│", "", ""]), the scrollbar becomes
invisible or bearly visible depending on the chosen character.
Assume that if the track symbol is empty, we expect the scrollbar
to be visible but not overlap with the pane separator.
Solution: Shift the horizontal position to the left.
This ensures the scrollbar remains visible and properly positioned
in all cases, matching the queue appareance.

oh yeah, that makes sense, thanks!


Update OpenAIWrapper to add fine tuning functionality with unit test cases.

Openai Reference:

check Fine-tuning steps in : https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updates.
full guide: https://platform.openai.com/docs/guides/fine-tuning/create-a-fine-tuned-model

Work scope:
The Openai API supports the ability to upload a json file with the training data and run fine-tuning functions on the data. Supporting this in intelliNode requires adding the following minimum functions to OpenAIWrapper.js:

upload fine tuning file.
create fine tune job.
use fine tune model.

Step1: Prepare the testing data in a file, for example:
{
  "messages": [
    { "role": "system", "content": "You are an assistant that occasionally misspells words" },
    { "role": "user", "content": "Tell me a story." },
    { "role": "assistant", "content": "One day a student went to schoool." }
  ]
}

Step2: Add a function inside OpenAIWrapper that match the following call to upload the file:
curl https://api.openai.com/v1/files \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -F "purpose=fine-tune" \
  -F "file=@path_to_your_file"

Step3: Add a function inside OpenAIWrapper that match the following call to create fine tuning hob:
curl https://api.openai.com/v1/fine_tuning/jobs \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENAI_API_KEY" \
-d '{
  "training_file": "TRAINING_FILE_ID",
  "model": "gpt-3.5-turbo-0613"
}'

Testing:
Add the testing scripts in OpenAIWrapper.test.js:

Test calling upload file.
Test calling create fine tune job.
Test calling the existing chat function with the fine tuned model name.

added fine tuning functionality, with unit tests
https://github.com/intelligentnode/IntelliNode/pull/48


seperate GetLeaderBoard and GetBoundedLeaderboard in game handler, ad…

…d integration test for game service

seperate GetLeaderboard and GetBoundedLeaderboard in game handler, add integration test for game service


[cherry-pick][branch-2.0][BugFix] enable-check-string-length (#8095)

What type of PR is this：

[x] bug
[ ] feature
[ ] enhancement
[ ] refactor
[ ] others

Which issues of this PR fixes ：

Fixes #8096
Problem Summary(Required) ：

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.16 out of 17 committers have signed the CLA.:white_check_mark: sevev:white_check_mark: decster:white_check_mark: Seaven:white_check_mark: stdpain:white_check_mark: gengjun-git:white_check_mark: chaoyli:white_check_mark: dirtysalt:white_check_mark: Astralidea:white_check_mark: RowenWoo:white_check_mark: xiaoyong-z:white_check_mark: meegoo:white_check_mark: imay:white_check_mark: Linkerist:white_check_mark: HangyuanLiu:white_check_mark: rickif:white_check_mark: trueeyu:x: mergify[bot]You have signed the CLA already but the status is still pending? Let us recheck it.


Prepare Core Libraries for April 2023 Release

API change check
API changes are not detected in this pull request.


Resizing PowerToys Run Search box

Description of the new feature / enhancement
Being able to change the searchbox size in PowerToys Run, to take up less screen space
Scenario when this would be used?
When i need to look up a file, i generally have to see the name of the file from another folder, resizing the whole search area, or atleast making the dimensions customizable at the user's end, can add a lot of functionality to its already high efficiency
Supporting information
No response

Thanks for the suggestion! Referencing https://github.com/microsoft/PowerToys/issues/12182 for consideration as we continue planning overall UX updates for the utility.

This has been worked on during the 0.77 Release sprint. Please update PowerToys to the latest release: https://github.com/microsoft/PowerToys/releases


DCGANs Tutorial's BatchNormalization is maybe something wrong

URL(s) with the issue:
https://www.tensorflow.org/tutorials/generative/dcgan
Description of issue (what needs changing):
https://colab.research.google.com/gist/MokkeMeguru/614e16d83d16f1eb70b5f3b73c7d070b/batchnormalization_debug.ipynb
In you tutorial, BatchNormalization will be Actnormalization in Glow(https://arxiv.org/abs/1807.03039)
Clear description
We need Correct BathNormalization
Usage example
We should input the shape when   BatchNormalization is initialized

Sorry, my misunderstanding about batch normalization.


task15 task16

@lakshmitrivenisareddy
Task -14
Code verified.
Task -15
Code verified.
Task -16
Code verified.
Task -17
Code verified.


Feature Model Anomaly + Subroutine Framework

@schweikart Thank you for the review, I've pushed a lot of tweaks

Everything should be ready now @schweikart :)

Oh, that's probably my bad.
SolverSelection would be more common as you can use these components just like any other:
<SolverSelection/> instead of getSolvers() (line 66)

One more thing:
This actually seems to cause problems! It resets the selection every time I try to select an option. Do you know what could cause this?

maybe use a Map instead? This typing will set all attributes of SolveMap instances to be SolveRequests but toString and other built-in attributes / functions are not SolveMaps!

I had a problem with Map<K,V> in the past, so I used this, but I can't replicate it now, so I'll use that instead.

I actually replicate the issue now. As Map<K, V> is an object, it gets formatted differently in the JSON that is sent to the backend.
The backend can't properly parse this format to a Java Map.


feat(satp-hermes): Fabric and Besu Bridges POC

Bridges POC

Test for Besu and Fabric Implementation
Besu and Fabric Contracts (non-standard ERC20 example and the wrapper contract)
Wrapper contracts now support ontologies (how to interact other contracts)

closes #3281

seems like a great job. Can you please remove unnecessary files such as soljson-v0.8.20+commit.a1b79de6.js and the generated .jsons? Instead of hardcoding them, include scripts to generate those (using the solc compiler or even better, foundry). You can find examples of the usage of foundry for example here https://github.com/RafaelAPB/data-transfer-dendreth

👍👍👍

seems like a great job. Can you please remove unnecessary files such as soljson-v0.8.20+commit.a1b79de6.js and the generated .jsons? Instead of hardcoding them, include scripts to generate those (using the solc compiler or even better, foundry). You can find examples of the usage of foundry for example here https://github.com/RafaelAPB/data-transfer-dendreth

@RafaelAPB
OK all done, included a script to use the forge plugin from foundry.


Automated Package Upgrade

:tada: This PR is included in version 7.7.46 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Code review FetchInboxCommand and FetchMessagesCommand

Update the code based on FetchInboxCommand code review.

1 - Move all the logic from fetch service to the fetch commands.
2 - Use fetch service just to handle the page and part repositories.
3 - Create the command slice to host all commands and move the fetch commands.
4 - Update the tests that uses FetchService to create the commands.
5 - Account service should start/stop fetch commands on account link/unlink
Acceptance criteria:
1 - Updated code merged.
2 - Screen capture of account link/unlink. Fetch starts on account link and stops on account unlink.
3 - All mentioned issues are closed with respective acceptance criteria proofs.

code merged in #58
missing screen capture

needs to build the UI for test these in the example app


fix(data toolbar): format groups correctly inside expandable content

Fixes needed for the Data Toolbar React component PR.

PatternFly-Next preview: https://patternfly-next-pr-2229.surge.sh

Just want to confirm the grid gap between __content and __expandable-content should be smaller than the gaps between the __expandable-content's?

I think that we should create a new example that shows the expandable-content class, aside from all of the other classes. It took me a while to find out which examples they were in. Maybe 1 example that has two components with expandable-content classes and one with pf-m-expanded applied and one without?
We can do it outside of the scope of this PR... @mattnolting @nicolethoen @mcoker what do you think? If so I can create a follow up issue

@christiemolloy I agree that we should do something to demonstrate the pf-m-expanded state in action. They could be added to the appropriate existing examples (toggle group, select filter, and stacked)

Just want to confirm the grid gap between __content and __expandable-content should be smaller than the gaps between the __expandable-content's?

NICE CATCH!
at line 7 can we update?
--pf-c-data-toolbar--GridGap: var(--pf-global--gutter);

I think that we should create a new example that shows the expandable-content class, aside from all of the other classes. It took me a while to find out which examples they were in. Maybe 1 example that has two components with expandable-content classes and one with pf-m-expanded applied and one without?

I agree, we should update the examples.

We can do it outside of the scope of this PR... @mattnolting @nicolethoen @mcoker what do you think? If so I can create a follow up issue

I'm open to either updating the existing examples in this PR, or opening a new issue. The primary purpose of this PR is to unblock @nicolethoen, so I'll leave that up to her..

@christiemolloy would you mind opening a new issue for the expanded demos so we can get this PR in ASAP?

:tada: This PR is included in version 2.31.5 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Use mount-s3 as a crate dependency

Tell us more about this new feature.
Hey, was wondering if in future it would be reasonable to consume mount-s3 as a crate. We currently install it as a package and then invoke the binary as a subprocess. One thing that would be improved by crate consumption would be error handling. Currently a mount error spits out a Rust error as a string and the exit code doesn't communicate error type.
I see that crates from this repository are already published but there's disclaimers.

Hey Jonathon, this is an interesting idea! While it's not something we're considering in-depth right now, it's something we discussed today as a possible way to help unblock some use cases like this.
For the error handling use case, we also have issue #721 to gather feedback about how we could improve error reporting/handling in Mountpoint. If there's anything specific you had in mind, we'd love to hear more.
I'm also curious if there are other customization you had in mind. Is it just error handling, or are there other parts you'd like to customize right now?


FlutterP2p.discoverDevices() not working

I use the example app and when I tapped the Discover Devices button, it threw an exception.
exception:
V/ContentValues(30506): Method: isLocationPermissionGranted
V/ContentValues(30506): Method: register
W/ter_p2p_exampl(30506): Accessing hidden field Ljava/nio/Buffer;->address:J (light greylist, reflection)
I/flutter (30506): stateChange: false
I/flutter (30506): connectionChange: false, Connected: false
I/flutter (30506): deviceChange: Android_14f9 / 1a:f0:e4:dd:26:ad / 10-0050F204-5 /  -GO
V/ContentValues(30506): Method: discover
E/flutter (30506): [ERROR:flutter/lib/ui/ui_dart_state.cc(157)] Unhandled Exception: PlatformException(2, null, null)
[38;5;244mE/flutter (30506): #0      StandardMethodCodec.decodeEnvelope[39;49m
[38;5;244mE/flutter (30506): #1      MethodChannel._invokeMethod[39;49m
E/flutter (30506): <asynchronous suspension>
[38;5;244mE/flutter (30506): #2      MethodChannel.invokeMethod[39;49m
[38;5;248mE/flutter (30506): #3      FlutterP2p.discoverDevices[39;49m
[38;5;248mE/flutter (30506): #4      _MyAppState.build.<anonymous closure>[39;49m
[38;5;244mE/flutter (30506): #5      _InkResponseState._handleTap[39;49m
[38;5;244mE/flutter (30506): #6      _InkResponseState.build.<anonymous closure>[39;49m
[38;5;244mE/flutter (30506): #7      GestureRecognizer.invokeCallback[39;49m
[38;5;244mE/flutter (30506): #8      TapGestureRecognizer.handleTapUp[39;49m
[38;5;244mE/flutter (30506): #9      BaseTapGestureRecognizer._checkUp[39;49m
[38;5;244mE/flutter (30506): #10     BaseTapGestureRecognizer.handlePrimaryPointer[39;49m
[38;5;244mE/flutter (30506): #11     PrimaryPointerGestureRecognizer.handleEvent[39;49m
[38;5;244mE/flutter (30506): #12     PointerRouter._dispatch[39;49m
[38;5;244mE/flutter (30506): #13     PointerRouter._dispatchEventToRoutes.<anonymous closure>[39;49m
[38;5;244mE/flutter (30506): #14     _LinkedHashMapMixin.forEach  (dart:collection-patch/compact_hash.dart:379:8)[39;49m
[38;5;244mE/flutter (30506): #15     PointerRouter._dispatchEventToRoutes[39;49m
[38;5;244mE/flutter (30506): #16     PointerRouter.route[39;49m
[38;5;244mE/flutter (30506): #17     GestureBinding.handleEvent[39;49m
[38;5;244mE/flutter (30506): #18     GestureBinding.dispatchEvent[39;49m
[38;5;244mE/flutter (30506): #19     GestureBinding._handlePointerEvent[39;49m
[38;5;244mE/flutter (30506): #20     GestureBinding._flushPointerEventQueue[39;49m
[38;5;244mE/flutter (30506): #21     GestureBinding._handlePointerDataPacket[39;49m
[38;5;244mE/flutter (30506): #22     _rootRunUnary  (dart:async/zone.dart:1138:13)[39;49m
[38;5;244mE/flutter (30506): #23     _CustomZone.runUnary  (dart:async/zone.dart:1031:19)[39;49m
[38;5;244mE/flutter (30506): #24     _CustomZone.runUnaryGuarded  (dart:async/zone.dart:933:7)[39;49m
[38;5;244mE/flutter (30506): #25     _invoke1  (dart:ui/hooks.dart:273:10)[39;49m
[38;5;244mE/flutter (30506): #26     _dispatchPointerDataPacket  (dart:ui/hooks.dart:182:5)[39;49m
E/flutter (30506):
V/ContentValues(30506): Method: unregister

Why does it happen ?

What's the output of flutter --version?

Result of flutter --version:
Waiting for another flutter command to release the startup lock...
Flutter 1.13.6 • channel beta • https://github.com/flutter/flutter.git
Framework • revision 659dc8129d (5 weeks ago) • 2019-12-30 09:24:47 -0800
Engine • revision bdc9708d23
Tools • Dart 2.8.0 (build 2.8.0-dev.0.0 c547f5d933)

Sorry for the delay. I'll take a look on that this week

Thanks a lot

I'm facing the same issue:
flutter --version
Flutter 1.12.13+hotfix.8 • channel stable • https://github.com/flutter/flutter.git
Framework • revision 0b8abb4724 (6 weeks ago) • 2020-02-11 11:44:36 -0800
Engine • revision e1e6ced81d
Tools • Dart 2.7.0

I'm facing same issue:
Flutter 1.21.0-6.0.pre.91 • channel master • https://github.com/flutter/flutter.git Framework • revision aeedd71e84 (5 hours ago) • 2020-07-29 03:41:03 -0400 Engine • revision a9910e409c Tools • Dart 2.10.0 (build 2.10.0-1.0.dev 24c7666def)
/flutter ( 6973): On Discover V/ContentValues( 6973): Method: discover E/flutter ( 6973): [ERROR:flutter/lib/ui/ui_dart_state.cc(167)] Unhandled Exception: PlatformException(2, null, null) E/flutter ( 6973): #0      StandardMethodCodec.decodeEnvelope  package:flutter/…/services/message_codecs.dart:572 E/flutter ( 6973): #1      MethodChannel._invokeMethod  package:flutter/…/services/platform_channel.dart:161 E/flutter ( 6973): <asynchronous suspension> E/flutter ( 6973): #2      MethodChannel.invokeMethod  package:flutter/…/services/platform_channel.dart:334 E/flutter ( 6973): #3      FlutterP2p.discoverDevices  package:flutter_p2p/plugin.dart:48 E/flutter ( 6973): #4      _MyAppState.build.<anonymous closure>  package:easyshare/main.dart:163 E/flutter ( 6973): #5      _InkResponseState._handleTap  package:flutter/…/material/ink_well.dart:992 E/flutter ( 6973): #6      _InkResponseState.build.<anonymous closure>  package:flutter/…/material/ink_well.dart:1101 E/flutter ( 6973): #7      GestureRecognizer.invokeCallback  package:flutter/…/gestures/recognizer.dart:183 E/flutter ( 6973): #8      TapGestureRecognizer.handleTapUp  package:flutter/…/gestures/tap.dart:523 E/flutter ( 6973): #9      BaseTapGestureRecognizer._checkUp

Hi @devtronic,
Any update on this?
Below are my flutter version details: flutter --version
Flutter 1.20.3 • channel stable • https://github.com/flutter/flutter.git
Framework • revision 216dee60c0 (5 days ago) • 2020-09-01 12:24:47 -0700
Engine • revision d1bc06f032
Tools • Dart 2.9.2

Maybe there is a fake issue, try to turn on your GPS, and restart the app. 😄

Maybe there is a fake issue, try to turn on your GPS, and restart the app. 😄


Better French i18n

Hello!
As a native French speaker I was amazed to see that Gherkin supports writing features in my mother tongue. But I was a bit bothered by the fact that some expressions that I thought should be keywords weren't, or didn't conform to the French grammar.
This PR aims at fixing those grammar issues.
In the process of making those fixes, I spotted an issue with the python pickles compiler regarding UTF-8 encoding and fixed it.
I did run the full test suite with
make clean all

but couldn't run the objective-c suite because my Linux box doesn't have Xcode... (I don't own an Apple computer and don't have easy access to one).
All the other tests pass!

Actually there were some more issues with the Python pickles compiler regarding UTF-8 encoding - related to Scenario Outlines in this case. Now they are also fixed https://github.com/cucumber/gherkin3/commit/46b79108cfabba11999c39d0a12ec5561de5e632.

Hi @moreau-nicolas,
An identical change was suggested a couple of years ago by @gregorybesson in cucumber/gherkin#223. I ended up rejecting it :crying_cat_face:
Reading your arguments and thinking about it I'm happy with this change.
Merci!

Hello @aslakhellesoy !
It's never too late :)
Congratz @moreau-nicolas : I think this is a great addition. Maybe I was rite too early :D
I'll be happy to benefit from this addition.
Greg

@aslakhellesoy You're welcome, I'm happy to help enhance an already great tool. :smiley:
@brasmusson, @gregorybesson Merci!

Hmm, I was unaware of the history of related, rejected PR's in gherkin2, and honestly mostly focused in getting in the in the fixes in the Python compiler here - and the related ones I tracked down. They were actually merged into the master branch separately. Rather than leaving this PR half merged, I merged all of it. I'm glad everything ended up OK.

I've come to realize that this change actually introduce kind of a backward compatibility problem. An existing feature with a step:
  Etant donné que j'aime les gâteaux

and a step definition with the regex:
  /^que j'aime les gâteaux$/

did work before but will not work after this change, since now the que will be considered as part of the keyword and needs to be removed from the step definition regex. This raises the question of the numbering of the next gherkin3 release (and the next Cucumber-Ruby release).

I've been a little worried about this too. It's true this would be a backwards incompatible change. It's weird to have gherkin3 version 4 though. I'd be happier to have gherkin version 4 if you see what I mean.
Maybe it's time to rename this repository (and the various packages) from gherkin3 to simply gherkin. And rename the old repo to gherkin2.
WDYT @brasmusson?

+1 to gherkin only. Having a version number in the name makes it confusing and hard to stick to semver. When following the semver rules, the major version number can climb quite quickly, often too quickly to have it included in the package name itself.

The problem is package names. I don't think the name gherkin is available on all platforms. Python I think.

IIRC one of the reasons of naming the ruby-gem from this project "gherkin3" was to enable the use of both the gherkin(2) gem and the gherkin3 gem, the first spike in Cucumber-Ruby did use both, didn't it. We have passed that for the ruby version. It is a matter of timing, renaming the repo now when Cucumber-JVM still used gherkin2 might lead to that some issues that should be raised on that repo is raised here instead (they wont be fixed either way). Renaming the released artifacts would block use of the parser from here, and for instance the tag expression handling for command lines from gherkin2. But where would that be a problem? In Cucumber-JVM? In Cucumber-JS?

Yes, ruby is not a problem any longer. Java and JavaScript should be fine too.
The main problem is that gherkin is already a thing on pypi: https://pypi.python.org/pypi/gherkin


Support type hinting

Add target
type-check: ## check types with mypy
	mypy -p package-name

Add dependency to requirments-dev.txt
Need to add mypy.

Duplicated with #101

Duplicated with #101


Can register source and register trigger be served by different sub domains?

Hey,
Is there a problem sending register source and register trigger to different sub domains?
Will Chrome be able to attribute the register source event to the register trigger event?
Amir

The reporting origin must match in order for the trigger to be attributed to the source.

Are you able to work around this requirement using redirects? For example, if you currently register sources at https://foo.mydomain.example/x and triggers at https://bar.mydomain.example/y you can have those URLs redirect to https://register.mydomain.example/source and https://register.mydomain.example/trigger. The reporting origin in that case will be https://register.mydomain.example.

@amirmatar Just following up on this.

I was able to make it work using redirects but I don't think this is a good solution because it duplicates the amount of calls between the browser and our servers. I believe many ad networks have different subdomains for click and conversion endpoints so either they add these redirects or they need to change their topology.
Can you please explain why can't Chrome attribute conversions to events based on the eTLD+1 domain?

We try to scope everything to origin for security reasons. If you're concerned that the redirect approach isn't performant enough, please consider filing an issue on https://github.com/WICG/attribution-reporting-api in which we can discuss alternative options.


New EFCore7 JSON columns not supported by BulkUpdate

Hello,
I recently started using the new JSON columns introduced by EFCore 7 : https://learn.microsoft.com/en-us/ef/core/what-is-new/ef-core-7.0/whatsnew#json-columns
Everything seems to be working smoothly on EFCore side but the BulkUpdate does not seem to support it.
Is it a known issue ? I could not find any ticket about it.
EfCore configuration looks simply like this :
builder.OwnsOne(e => e.MyJsonColum, navigationBuilder => navigationBuilder.ToJson());
When spying SQL request, I see that a SELECT is generated with empty fields : SELECT TOP 0 T.[Id], T.[], T.[], T.[], T.[], T.[] FROM INTO [dbo].[MyTableTemp8c9e6f45] FROM [dbo].[MyTable] AS T LEFT JOIN [dbo].[MyTable] AS Source ON 1 = 0; (removed useless data)
Best regards,

I have the same issue

should now work with 7.1.3


Update SharpDX

As a maintenance task, would you accept a PR where SharpDX was updated to 3.1.1?

Hi,
Thank you for your offer.
ATF is maintained in a provide repository, because of that applying PRs is time consuming.
At this time only major bugs are fixed.
Thanks
Alan Beckus


Deleting spam once again...

Deleting spam once again...

Why I should want?
вт, 18 февр. 2020 г. в 1:13, heken375<EMAIL_ADDRESS>
Hello!😻😻😻 Wanna see my private photos?💋💋💋 Go to this site and find
me here. ❤❤❤ My nick is pussycat18 👇👇👇
https://tinyurl.com/rupocj5
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/trekhleb/javascript-algorithms/issues/464?email_source=notifications&email_token=AJHRNVLQIPOW3DZO5BR7NHTRDLOVDA5CNFSM4KWXEJ32YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IODXFRQ,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJHRNVOQVSCIFDJK5HIVR4TRDLOVDANCNFSM4KWXEJ3Q
.


Negative decimal numbers not parsed correctly

Came here via rasterio calc. Apparently snuggs does not parse negative decimal numbers correctly.
Below statement returns False:
snuggs.eval("(< -0.9 0)")

Comparing with a whole negative number returns True:
snuggs.eval("(< -1 0)")

Apparently the parser misses the sign when parsing a decimal number above -1. Compare:
snuggs.eval("(+ -0.9 0)")
snuggs.eval("(+ -1.7 0)")

@tomvansteijn thanks for the report! I'm now using hypothesis to hammer on the number parsers and found another bug, also fixed. 1.4.4 is on PyPI now and has both the fixes.