Telestra App Note

Resolving Network Issues between the Target and ACE Studio Project Manager (#92)

This application note covers the following common network issues:

  • Target and its projects are missing from the ACE Studio Open Projects Menu List
  • Target shows mDNS error
  • Project fails to open
  • Project fails to install
  • Project fails to save
  • Project fails to install on boot-up
  • Project fails to install when run from command line/SSH
  • Project fails to copy/clone from one Target to another via the Remote Management System (RMS)

In order for ACE Studio, Target, Project Install and RMS to function correctly the network configuration must be setup properly. If the network configuration is not setup properly, it is easy for a user to run into an error message that says, “'hg', 'clone', 'https://target-name/project' returned non-zero exist status 255”. The most common place for this to occur is in ACE Studio when trying to open up a project, however it can also occur when trying to copy/clone a project in RMS or when the default project is installed during boot up. This application note provides possible causes and solutions to resolving this error.

Error Message

The error message seen will be similar to what you see below:

Command '['hg' , 'clone', 'https://RC-Quad-Core.local/projects/737ng-1.17'.' 
/tmp/ace-projects-aceuser/737ng-1.17_k954Xc/737ng-1.17']' 
returned non-zero exit status 255 

The error message appears inside of ACE Studio, RMS, or in the log file on a default install on boot-up.

Click image to enlarge


Possible Causes of the Error Message

There are several reasons why this message to appears. Generally, the cause is network related; however it could also be a disk space issue, multiple interfaces on the same physical switch, Master/Slave Target issue, etc.

In order to clear up this error EVERY possible cause should be investigated and resolved. All of the known causes are shown in the table below. Although the cause will vary from configuration to configuration ASTi recommends going through the list below one-by-one until this error message goes away and the Project Management is behaving normal. Continue reading below the table for details on each solution.

Possible Error Cause Solution
Incorrect Network Settings
  • Verify Target Network Settings and reboot
  • Verify Studio Network Settings and reboot
  • Perform a "hostname.local" ping
Studio Virtual Machine not in Bridged Mode Change the VM to operate in Bridged mode
Hard Disk is Full
  • Verify Hard Disk is full
  • Clean up necessary partition
Two Target Ethernet interfaces are connected to the same physical LAN Switch Use separate Switches or VLANs
Master Target is not up and running Fix Master Target or break Slave/Master relationship
Studio and Target are on Different Subnets Copy the URL (with IP instead of host name) and paste in Open URL
Cloned Image resulting in Duplicate MAC addresses in VM Studio Generate a new MAC address for the Virtual Machine

Network Settings
  1. Verify that the Telestra Target Network Settings are setup correctly using RMS. The following must be setup correctly at a minimum:

    Mode = Fixed or DHCP, DHCP requires a DHCP Server (non-ASTi)

    IPv4 Address = valid IP address
    Important: You cannot have duplicate IP addresses on the network.

    Subnet Mask = valid subnet mask for the above address

    Default Route = eth0, eth2, etc.

    Domain is only specified if there is a valid domain, otherwise leave blank.

    Gateway IP is only specified if there is a gateway IP address, otherwise leave blank.

    Hostname must be configured.

    Nameserver is only specified if there is a valid nameserver, otherwise leave blank.

    So at a minimum you need an IPv4 address, subnet mask, default route and hostname. Reboot after any changes.

    Click image to enlarge

    Click image to enlarge

  2. Verify that the ACE Studio platform (or VM) Network Settings are setup correctly. This is done through the operating system (RHEL5 GUI).

    The following must be setup correctly at a minimum:

    Mode = Fixed or DHCP, DHCP requires a DHCP Server (non-ASTi)

    class="strong>IPv4 Address = valid IP address
    Important: You cannot have duplicate IP addresses on the network.

    Subnet Mask = valid subnet mask for the above address

    Domain is only specified if there is a valid domain, otherwise leave blank.

    Gateway IP is set to the gateway IP address. If no gateway exists set this to match the IPv4 address above. The Studio platform must have a gateway IP address.

    Hostname must be configured.

    Nameserver/DNS is only specified if there is a valid nameserver, otherwise leave blank.

    Domain is only specified if there is a valid domain, otherwise leave blank.

    So at a minimum you need an IPv4 address, subnet mask, default gateway and hostname. Reboot after any changes.





  3. Ensure that ACE Studio and the Target are on the same subnet. For example if you have a Class C subnet then ACE Studio might have the following settings: 192.168.100.1/24 and the Target might be something like 192.168.100.2/24. However, if the Target was on 192.168.101.2/24 then things will not work as expected. So ensure both platforms are on the same subnet.
  4. Ensure that there are no duplicate Target/Studio IP addresses on the network. The 'arping' command may be useful to debug this problem. For example:

    [root@exampleradios ~]# arping -I eth0 -c 3 10.2.0.162
    ARPING 10.2.0.162 from 10.2.0.162 eth0
    Sent 3 probes (3 broadcast(s))
    Received 0 response(s)
    

    If there was a duplicate IP address you will see the following:

    [root@exampleradios ~]# arping -D -I eth0 -c 3 10.2.0.165
    ARPING 10.2.0.165 from 0.0.0.0 eth0
    Unicast reply from 10.2.0.165 [00:0C:29:F8:48:E6] 
    for 10.2.0.165 [00:0C:29:F8:48:E6] 1.136ms
    Sent 1 probes (1 broadcast(s))
    Received 1 response(s)
    
  5. Try to ping from ACE Studio to the Target. There are two methods for this. First, ping from ACE Studio to the Target using the Target's IP address. Second, ping from ACE Studio to the Target using the hostname.local address. Confirm that you have at least 30 successful pings in a row before terminating.

    For example:

    # ping 10.2.0.144
    
    # ping my-target-hostname.local
    

    If the IP address ping fails then there is something fundamentally wrong with the network setup. Go back and check cabling, switches and all network settings.

    If the hostname.local ping fails continue on with this application note before trying to open a project.


ACE Studio Virtual Machine not in Bridged Mode

Configure the Virtual Machine (VM) to operate in bridged mode. Additionally, verify that the PC that the VM is running on is on the same subnet as the Studio VM and the Telestra Target.

If this error occurs when running ACE Studio on a Virtual Machine (Studio VM) it is most likely a networking issue or potentially an mDNS problem. For Studio VM to properly function the host's networking must be properly configured as well as the VM's networking. Due to the way ACE Studio communicates with Targets, it is possible for ACE Studio to list Targets and Projects correctly but not be able to open them. Target and Project information publication is all handled with multicast messages over IPv6 which can function even when the host machine or VM are configured incorrectly. In other words, the Target and ACE Studio may see each other via mDNS multicast, however when you go to open a Project via IPv4 it fails.

Check to make sure that you are using Bridged mode on the VM network interface. Bridged mode allows your VM to talk to the outside world via your host machine's network card by connecting your virtual network card to the physical network. NAT mode just doesn't seem to work as well due to the network IP sharing. NAT mode allows the VM to share the IP address of the host system, which is specifically useful for wireless Ethernet networking. This IP address sharing can cause problems with ACE Studio functionality.


Hard Disk is Full

If the Telestra Target hard drive is full then the Project will not load and the clone error will occur. To check the current status of the hard-drive utilization go to the Telestra RMS Health System:

RMS > Health > Platform Info > Disks

Click image to enlarge

Verify that no partition is more than 90% utilized, if a partition is over 90% then some file-cleanup is necessary. The most likely cause is that the /var/log/messages file(s) have grown very large in size.


Two Target Ethernet interfaces are connected to the same physical LAN Switch

A problem occurs if you try to connect two of the Target interfaces to the same physical LAN Switch that the Studio is also on. In this case the Studio platform will see packets from both interfaces and get confused as to which IP address to resolve for hostname.local. In this case if you ping hostname.local from Studio you may see that the ping command works 50% of the time and fails about 50% of the time.

The solution for this is to ensure that only one Target interface can see the Studio interface. After this is completed reboot both platforms to ensure that the system functions correctly.


Bad Target/Switch Connection


Correct Target/Switch Connection


Master Target is Not Up and Running

If you are using the Multi-Target Project Management feature then you will get this error message if you try and open a Slave Project when the Master Project has failed or is not running. To resolve this ensure that the Master Target and project are running correctly. Or delete the Master Target Project link via RMS and manage the project locally. For further information on this please refer to Application Note 90.


Studio and Target are on Different Subnets

If the targetname.local is failing due to the Studio and Target being on different networks, using the following Open URL instructions will provide a work around. In RMS navigate to Project Management > Show Projects. Open the project you want to use. Copy the URL (with the IP address instead of the host name) in the title bar and paste it into ACE Studio under Project > Open URL.

In ACE Studio, enter the project's URL. For example, https://10.2.121.10/projects/<Project Name>.

In order to install the project the Target name must be deleted from the Telestra Editor. Clear the name by opening the project, right-click on the Telestra in the layout and select edit, and delete the field that says 'Target'.


Cloned Image resulting in Duplicate MAC addresses in VM Studio

If a computer image is cloned from a master image on another computer, the cloned image will use the MAC address of the original computer. This results in duplicate MAC addresses on the network.

Generate a new MAC address by opening the Virtual Machine (VM) file and in the top menu bar select Virtual Machine -> Settings -> Network -> Advanced Options. Select the "Generate" button to create a new MAC address which should appear in the field. The VM file is ready to start with the new MAC Address.